| Time |
Nickname |
Message |
|
00:01
🔗
|
|
svchfoo1 has joined #urlteam |
|
00:02
🔗
|
|
svchfoo3 sets mode: +o svchfoo1 |
|
00:03
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
00:04
🔗
|
|
dashcloud has joined #urlteam |
|
00:05
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
00:30
🔗
|
|
dashcloud has quit IRC (Read error: Connection reset by peer) |
|
00:31
🔗
|
|
dashcloud has joined #urlteam |
|
00:31
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
01:00
🔗
|
|
Silvan has quit IRC (Read error: Operation timed out) |
|
01:02
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
01:06
🔗
|
|
dashcloud has joined #urlteam |
|
01:06
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
01:07
🔗
|
|
bwn_ has quit IRC (Read error: Operation timed out) |
|
01:10
🔗
|
|
bwn has joined #urlteam |
|
01:28
🔗
|
|
JesseW has joined #urlteam |
|
01:28
🔗
|
|
svchfoo1 sets mode: +o JesseW |
|
01:33
🔗
|
JesseW |
up'ed the queue for da-gd to 10 items |
|
01:34
🔗
|
JesseW |
7 results found so far, in ~ 120,000 checks |
|
01:34
🔗
|
JesseW |
hm, if they were evenly distributed, we should have found more like 12 by now |
|
01:34
🔗
|
JesseW |
apparently they aren't |
|
01:39
🔗
|
JesseW |
I think we have gotten everything from dld-bz |
|
01:39
🔗
|
JesseW |
I've stopped it for now |
|
01:52
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
01:54
🔗
|
|
dashcloud has joined #urlteam |
|
01:55
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
02:22
🔗
|
JesseW |
started ccl.hu -- we'll see if it picks up *anything* useful |
|
02:44
🔗
|
JesseW |
starting chilp-it -- 6 (and 7) character non-incremental lowercase alpha & digits. So a search space of about 2 billion for the 6 character ones, and 78 billion for the 7 character ones. |
|
02:45
🔗
|
JesseW |
we can certainly do the 6 character ones; maybe not the 7 characters |
|
02:47
🔗
|
aaaaaaaaa |
go big or go home |
|
02:48
🔗
|
JesseW |
I mean, we can certainly *start* the 7 character ones -- but considering in about a year's running we've only done about 21 billion checks total... |
|
02:50
🔗
|
aaaaaaaaa |
Well, I was referring to the whole idea that you can't finish things you don't start. But in all seriousness, the tracker might have to be a bit more robust to handle the traffic necessary for a grab that big. |
|
02:50
🔗
|
JesseW |
and we'd need to boost the number of people running warriors by quite a bit |
|
02:54
🔗
|
|
W1nterFox has quit IRC (Ping timeout: 506 seconds) |
|
03:00
🔗
|
JesseW |
sadly, it looks like ccl.hu died before we could work on it. :-( |
|
03:01
🔗
|
JesseW |
100,000 checks and nothing useful found. |
|
03:08
🔗
|
|
W1nterFox has joined #urlteam |
|
03:35
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
|
03:37
🔗
|
|
winterfox has joined #urlteam |
|
03:40
🔗
|
|
W1nterFox has quit IRC (Read error: Operation timed out) |
|
04:12
🔗
|
JesseW |
So there are 6 url shortening services that signed up with 301works, but have since died. We need to ping IA (and the other folks involved with 301works) and get the material un-darked. |
|
04:21
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
04:24
🔗
|
|
dashcloud has joined #urlteam |
|
04:25
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
04:51
🔗
|
|
aaaaaaaaa has quit IRC (Leaving) |
|
05:23
🔗
|
|
Mayonaise has quit IRC (Ping timeout: 606 seconds) |
|
05:23
🔗
|
|
zhongfu_ has joined #urlteam |
|
05:24
🔗
|
|
dashcloud has quit IRC (Ping timeout: 606 seconds) |
|
05:24
🔗
|
|
dashcloud has joined #urlteam |
|
05:24
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
05:26
🔗
|
|
Mayonaise has joined #urlteam |
|
05:27
🔗
|
|
zhongfu has quit IRC (Ping timeout: 606 seconds) |
|
05:33
🔗
|
JesseW |
chfoo -- it'd be nice to get this PR deployed sooner than later, as it would stop migre-me from cluttering up the error_reports, and having to skip a bunch of items (which we'll have to go back through and catch later) |
|
05:34
🔗
|
JesseW |
and apparently something broke the error reports display :-( |
|
05:35
🔗
|
chfoo |
_tt_tmp = reverse_url('admin.error_reports')+'?project_id=' + report['project'] # admin/overview/error_reports.html:40 (via admin/base.html:25, base.html:13) |
|
05:35
🔗
|
chfoo |
TypeError: Can't convert 'NoneType' object to str implicitly |
|
05:36
🔗
|
JesseW |
lol. :-( |
|
05:36
🔗
|
JesseW |
PR coming asap |
|
05:36
🔗
|
chfoo |
does the anti regex need more code? does doing (http://example.com/)|(^$) make sense? |
|
05:37
🔗
|
chfoo |
or perhaps ^\s*$ |
|
05:38
🔗
|
JesseW |
I don't understand what you mean...? Yes, a good way to enable the tolerate_missing_Location feature would be ^$ if that's the only pattern you want to match, or something_else.php|^$ if you wanted to match both... |
|
05:38
🔗
|
JesseW |
but I don't think it needs "more code"...? |
|
05:39
🔗
|
chfoo |
i mean why does it need a pull request for more logic? i can't see why the existing code won't work with adding ^$ to the option |
|
05:42
🔗
|
JesseW |
ah, because it doesn't get into the existing branch if there is no Location header at all |
|
05:44
🔗
|
chfoo |
oh, ok. i see it now |
|
05:45
🔗
|
JesseW |
it works if the Location header is present, but empty |
|
05:45
🔗
|
JesseW |
it (already) works, I mean |
|
05:45
🔗
|
chfoo |
right |
|
05:46
🔗
|
chfoo |
ok, seems good. you can merge and let me know when you have the error page thing fixed |
|
05:46
🔗
|
chfoo |
and i'll update the code on the tracker |
|
05:47
🔗
|
JesseW |
ok |
|
05:52
🔗
|
|
ahrain has quit IRC (Remote host closed the connection) |
|
05:59
🔗
|
JesseW |
chfoo: as GitHub83 mentioned, I've pushed a fix now. |
|
06:00
🔗
|
chfoo |
ok, updating the tracker |
|
06:01
🔗
|
JesseW |
excellent, thanks |
|
06:02
🔗
|
|
xmc has joined #urlteam |
|
06:02
🔗
|
|
swebb sets mode: +o xmc |
|
06:04
🔗
|
JesseW |
let me know when the tracker comes back up |
|
06:08
🔗
|
chfoo |
JesseW: everything should be ok now |
|
06:08
🔗
|
|
GitHub97 has joined #urlteam |
|
06:08
🔗
|
|
GitHub97 has left |
|
06:08
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
06:09
🔗
|
JesseW |
cool, looks good |
|
06:12
🔗
|
|
dashcloud has joined #urlteam |
|
06:12
🔗
|
|
svchfoo3 sets mode: +o dashcloud |
|
06:35
🔗
|
JesseW |
Atluxity: getting close to the top 10! Only 31 million scans to go. :-) |
|
06:46
🔗
|
Atluxity |
woho :D |
|
06:51
🔗
|
JesseW |
of course, you still have more than 3 billion scans to reach the top spot... ;-/ |
|
06:51
🔗
|
|
cechk01 has quit IRC (Read error: Connection reset by peer) |
|
06:59
🔗
|
Atluxity |
I think I'm gaining in... :P |
|
06:59
🔗
|
JesseW |
yeah, he doesn't seem to be active lately |
|
07:08
🔗
|
cybersec |
How long has this project been running for? |
|
07:08
🔗
|
cybersec |
were those 3.7 billion scans over the course of months/years, or has it only been going for a little bit? |
|
07:08
🔗
|
cybersec |
also has he ever popped in IRC before? |
|
07:09
🔗
|
cybersec |
scratch that, just realized he's in the channel right now haha |
|
07:09
🔗
|
JesseW |
Ah, terroroftinytown, the current tracker, is only a year old. |
|
07:10
🔗
|
JesseW |
Specifically, the earliest daily dumps from it were on Nov 6, 2014. |
|
07:10
🔗
|
JesseW |
Before it, there were other tools used to produce the old dump, but they aren't reflected in the tracker totals, AFAIK. |
|
07:11
🔗
|
cybersec |
ah, I see |
|
07:11
🔗
|
cybersec |
is there anything known about how johtso scans so crazily fast? haha |
|
07:12
🔗
|
JesseW |
cybersec: the rumor I heard is he was running it on the side of bitcoin mining equipment |
|
07:12
🔗
|
cybersec |
oh yeah! sorry, I forgot you told me that the other day |
|
07:12
🔗
|
* |
JesseW is going to sleep now. G'night, all -- if the tracker blows up, please put out the fire. |
|
07:13
🔗
|
cybersec |
that's pretty awesome |
|
07:13
🔗
|
cybersec |
night! |
|
07:23
🔗
|
|
JesseW has quit IRC (Leaving.) |
|
08:46
🔗
|
|
bwn has joined #urlteam |
|
10:00
🔗
|
|
asdf has quit IRC (Ping timeout: 252 seconds) |
|
10:51
🔗
|
winterfox |
I dont see why johtso's setup would need bitcoin mining hardware. From what I can tell you just need lots of IP addresses so shorteners dont block you for spamming them |
|
11:02
🔗
|
Atluxity |
a mining operation implies to me a small server-park |
|
11:02
🔗
|
Atluxity |
perhaps with public ip's |
|
11:44
🔗
|
|
cybersec has quit IRC (Ping timeout: 483 seconds) |
|
13:10
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
13:12
🔗
|
|
dashcloud has joined #urlteam |
|
13:13
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
14:27
🔗
|
|
winterfox has quit IRC (Remote host closed the connection) |
|
15:15
🔗
|
|
cybersec has joined #urlteam |
|
15:24
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
15:27
🔗
|
|
dashcloud has joined #urlteam |
|
15:27
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
15:33
🔗
|
|
cybersec has quit IRC (Quit: Sorry, fuckers.) |
|
15:36
🔗
|
|
dashcloud has quit IRC (Ping timeout: 252 seconds) |
|
15:36
🔗
|
|
dashcloud has joined #urlteam |
|
15:36
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
15:49
🔗
|
phuzion |
JW_work: I think I might have figured out why we're seeing a less-than-even distribution of urls on da.gd |
|
15:50
🔗
|
|
cechk01 has joined #urlteam |
|
15:52
🔗
|
phuzion |
Basically, there's some number of shorturls, greater than 30,000, that are "disabled" meaning they return 404 despite being "valid" URLs. Basically, da.gd was the victim of a spambot attack at some point, and my buddy disabled the URLs that were leading to viagra spam sites. |
|
15:52
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
17:05
🔗
|
jornane |
Hi! I just got a report from my organisations CERT. They have logged that my Warrior instance (which runs there with permission, naturally) is performing DNS lookups that are associated with malware. |
|
17:06
🔗
|
Atluxity |
did they specify? |
|
17:07
🔗
|
jornane |
The mail said "..bigfoot1942.sektori.org….." without quotes, it is part of a suricata log. |
|
17:07
🔗
|
Atluxity |
strange |
|
17:08
🔗
|
jornane |
Does the warrior check URLs it finds? |
|
17:12
🔗
|
|
bwn has quit IRC (Read error: Connection reset by peer) |
|
17:12
🔗
|
|
bwn has joined #urlteam |
|
17:13
🔗
|
|
Start has joined #urlteam |
|
17:16
🔗
|
phuzion |
jornane: I don't believe that the warrior does anything along those lines, but let me pcap a warrior running urlteam for a minute and see |
|
17:29
🔗
|
|
JesseW has joined #urlteam |
|
17:29
🔗
|
|
svchfoo1 sets mode: +o JesseW |
|
17:39
🔗
|
phuzion |
jornane: It doesn't appear that the urlteam scripts do anything like a dns lookup or http request to the actual site itself, just to the url shortener. I could be wrong, though. |
|
17:40
🔗
|
JesseW |
they do not touch the actual site, that's correct. |
|
17:41
🔗
|
jornane |
phuzion: I just started DNS logging myself with a Suricata of my own, so hopefully I'll find out where this comes from. Since the warrior was my latest addition to the network, I suspected it first. Looks like (hope ;-)) I was wrong. |
|
17:42
🔗
|
phuzion |
jornane: Your CERT team said that the malicious DNS lookup was from that warrior's IP address? Does that VM share an IP with anything/anyone else? |
|
17:42
🔗
|
JesseW |
but DNS queries are not the same as contacting the site. Has your warrior been running any other project besides urlteam? |
|
17:44
🔗
|
|
JesseW has quit IRC (Leaving.) |
|
17:46
🔗
|
jornane |
phuzion: yes, but it's even harder to explain the requests from the other hosts sharing the IP |
|
17:46
🔗
|
jornane |
Most of my projects run IPv6-only, only very special projects that require IPv4 do get access to my NAT |
|
17:46
🔗
|
phuzion |
Do any of the hosts sharing the IP run windows? |
|
17:46
🔗
|
jornane |
No |
|
17:48
🔗
|
phuzion |
I'd pcap the vm host (so you can figure out which VM it came from) and ask your CERT team to give you time and date stamps of any further hits so you can narrow it down. |
|
17:51
🔗
|
jornane |
phuzion: doing that now, so far the results look OK, just that during the last quarter the warrior has looked up youtube.com and twitter.com, those aren't listed on http://archiveteam.org/index.php?title=Urlteam |
|
17:51
🔗
|
phuzion |
https://github.com/ArchiveTeam/terroroftinytown-client-grab/blob/master/pipeline.py#L42-L47 |
|
17:52
🔗
|
jornane |
check, then it makes sense :) I saw the cheeseburger as well but assumed it had a redirect service :P |
|
18:04
🔗
|
|
JW_work1 has joined #urlteam |
|
18:04
🔗
|
|
JW_work has quit IRC (Read error: Operation timed out) |
|
18:19
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
18:20
🔗
|
|
dashcloud has joined #urlteam |
|
18:20
🔗
|
|
svchfoo1 sets mode: +o dashcloud |
|
18:25
🔗
|
|
cechk01 has quit IRC (Read error: Connection reset by peer) |
|
18:36
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
19:02
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
|
19:11
🔗
|
|
JW_work1 has quit IRC (Leaving.) |
|
19:12
🔗
|
|
JW_work has joined #urlteam |
|
19:22
🔗
|
|
Start has joined #urlteam |
|
19:23
🔗
|
|
bwn has joined #urlteam |
|
19:38
🔗
|
|
aaaaaaaaa has joined #urlteam |
|
19:38
🔗
|
|
swebb sets mode: +o aaaaaaaaa |
|
20:06
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
20:48
🔗
|
|
Start has joined #urlteam |
|
20:48
🔗
|
|
Start has quit IRC (Client Quit) |
|
21:04
🔗
|
|
Start has joined #urlteam |
|
21:10
🔗
|
|
asdf has joined #urlteam |
|
21:41
🔗
|
|
aaaaaaaa_ has joined #urlteam |
|
21:41
🔗
|
|
aaaaaaaaa has quit IRC (Read error: Connection reset by peer) |
|
21:41
🔗
|
|
swebb sets mode: +o aaaaaaaa_ |
|
21:44
🔗
|
|
aaaaaaaa_ is now known as aaaaaaaaa |
|
22:16
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
23:24
🔗
|
asdf |
this url shortener looks mysterious https://soli.dm/ |
|
23:24
🔗
|
asdf |
lol |
|
23:38
🔗
|
JW_work |
nice big letters |
|
23:38
🔗
|
JW_work |
feel free to add it anyway |
|
23:39
🔗
|
|
Start has joined #urlteam |
|
23:44
🔗
|
|
WinterFox has joined #urlteam |