| Time |
Nickname |
Message |
|
01:23
🔗
|
hook54321 |
http://budurl.com/uukc |
|
01:39
🔗
|
|
treyo has joined #urlteam |
|
01:45
🔗
|
|
HCross has quit IRC (Read error: Connection reset by peer) |
|
01:45
🔗
|
|
HCross has joined #urlteam |
|
01:50
🔗
|
|
treyo has quit IRC (Quit: Page closed) |
|
04:00
🔗
|
Somebody2 |
Sorry I've been out-of-contact for some time. Nice to see the go.usa.gov folks (treyo) speaking up, and being interested in archiving. |
|
04:00
🔗
|
Somebody2 |
I wondered how long it would be before they noticed how hard we were banging on their server. |
|
04:01
🔗
|
Somebody2 |
Re: x.vu -- I'll set it running for now; we might as well get what we can. |
|
04:16
🔗
|
Somebody2 |
x-vu started |
|
04:18
🔗
|
Somebody2 |
gg-gg hasn't gotten results lately; turning it off |
|
04:19
🔗
|
Somebody2 |
go-usa-gov is still working fine; I'll keep it going till treyo (or someone else) actually asks us to turn it off, or provides a better alternative. |
|
04:20
🔗
|
Somebody2 |
vgd_6 hasn't found anything recently, but there are only a million total results right now, so it's probably still fine. |
|
04:21
🔗
|
Somebody2 |
x-vu has gotten some results |
|
04:21
🔗
|
Somebody2 |
specifically, about 800 results so far |
|
04:22
🔗
|
Somebody2 |
x-vu returns HTTP 410 sometimes; added that as a no-redirect expected result |
|
04:23
🔗
|
Somebody2 |
boosting the queue to 30 |
|
04:24
🔗
|
Somebody2 |
Hm, I'm not sure if it's case sensitive or not |
|
04:24
🔗
|
Somebody2 |
the two character ones do not seem to be |
|
04:25
🔗
|
Somebody2 |
finished all of them, in any case |
|
04:26
🔗
|
Somebody2 |
boosting queue to 60 |
|
04:29
🔗
|
Somebody2 |
Interestingly, all of these seem to redirect to a warning page on xdotvu.com |
|
04:29
🔗
|
Somebody2 |
but the URL includes the real target, so it's good enough for our purposes |
|
04:32
🔗
|
Somebody2 |
getting a few timeouts, but ... it's going away soon. So, queue up to 90 |
|
04:34
🔗
|
Somebody2 |
we seem to have about 90 total warriors right now; so I'll boost the queue to 100, that way everybody can join in |
|
04:42
🔗
|
Somebody2 |
Finished the initial-digit-three-character ones. |
|
04:42
🔗
|
Somebody2 |
Hm, bunch of errors, dropping queue down to 90 |
|
05:09
🔗
|
Somebody2 |
checked over 100,000; nearly 4,000 found. |
|
05:46
🔗
|
Somebody2 |
cleared out the errors, now reloading the queue with 60 |
|
05:54
🔗
|
Somebody2 |
errors came back; draining queue, then will try 40 |
|
06:02
🔗
|
Somebody2 |
40 seems to work, trying 50 |
|
08:20
🔗
|
|
dashcloud has quit IRC (Read error: Operation timed out) |
|
08:20
🔗
|
|
dashcloud has joined #urlteam |
|
09:48
🔗
|
JAA |
Somebody2: As far as I can tell, the paid short (less than 3 characters) codes are case-insensitive, but the automatic six-character chodes are case-sensitive. No idea about the ones you can set yourself in the advanced options. |
|
10:10
🔗
|
JAA |
"terroroftinytown.client.errors.ScraperError: Number of attempts exceeded for 5708844300 (0-DHxu)." Hmm. |
|
10:18
🔗
|
|
T31M has joined #urlteam |
|
10:20
🔗
|
|
T31M has quit IRC (Leaving) |
|
14:05
🔗
|
|
zhongfu_ has joined #urlteam |
|
14:05
🔗
|
|
zhongfu has quit IRC (Ping timeout: 260 seconds) |
|
15:19
🔗
|
|
Jonison has joined #urlteam |
|
15:27
🔗
|
|
Jonison has quit IRC (Ping timeout: 260 seconds) |
|
15:40
🔗
|
Somebody2 |
I turned *off* gg-gg... |
|
15:42
🔗
|
Somebody2 |
x-vu has finished the 3-character ones |
|
15:43
🔗
|
Somebody2 |
and it looks like go-usa-gov has blocked us |
|
15:46
🔗
|
Somebody2 |
I'm trying to drain the queue on it, then we'll leave it off for a few days |
|
15:47
🔗
|
astrid |
assholes |
|
15:47
🔗
|
astrid |
:P |
|
15:47
🔗
|
Somebody2 |
Eh, go-usa-gov is fine; they politely came in a day or so ago, and asked about setting up a bulk export instead. |
|
15:48
🔗
|
Somebody2 |
I just thought I'd keep the scraper running till they actually told us to stop. |
|
15:48
🔗
|
astrid |
yeah |
|
15:51
🔗
|
Somebody2 |
eh, it doesn't seem to be draining; I'll just reset the autoqueue back to the last result, and clear it |
|
15:51
🔗
|
Somebody2 |
that way it won't interfere with the other jobs |
|
15:52
🔗
|
Somebody2 |
ok, going afk for the day |
|
15:52
🔗
|
astrid |
toodleoo |
|
20:26
🔗
|
|
svchfoo1 has quit IRC (Remote host closed the connection) |
|
20:26
🔗
|
|
svchfoo3 has quit IRC (Remote host closed the connection) |
|
20:27
🔗
|
|
svchfoo3 has joined #urlteam |
|
20:27
🔗
|
|
svchfoo1 has joined #urlteam |
|
20:29
🔗
|
|
JAA sets mode: +o svchfoo1 |
|
20:30
🔗
|
|
svchfoo1 sets mode: +o svchfoo3 |
|
20:39
🔗
|
|
astrid sets mode: +ooo joepie91_ HCross2 HCross |
|
20:52
🔗
|
|
svchfoo3 has quit IRC (Remote host closed the connection) |
|
20:53
🔗
|
|
Aoede has left WeeChat 1.9 |
|
20:53
🔗
|
|
svchfoo3 has joined #urlteam |
|
20:54
🔗
|
|
svchfoo1 sets mode: +o svchfoo3 |