#urlteam 2017-09-13,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
hook54321http://budurl.com/uukc [01:23]
.... (idle for 16mn)
***treyo has joined #urlteam [01:39]
HCross has quit IRC (Read error: Connection reset by peer)
HCross has joined #urlteam
[01:45]
treyo has quit IRC (Quit: Page closed) [01:50]
........................... (idle for 2h10mn)
Somebody2Sorry I've been out-of-contact for some time. Nice to see the go.usa.gov folks (treyo) speaking up, and being interested in archiving.
I wondered how long it would be before they noticed how hard we were banging on their server.
Re: x.vu -- I'll set it running for now; we might as well get what we can.
[04:00]
.... (idle for 15mn)
x-vu started
gg-gg hasn't gotten results lately; turning it off
go-usa-gov is still working fine; I'll keep it going till treyo (or someone else) actually asks us to turn it off, or provides a better alternative.
vgd_6 hasn't found anything recently, but there are only a million total results right now, so it's probably still fine.
x-vu has gotten some results
specifically, about 800 results so far
x-vu returns HTTP 410 sometimes; added that as a no-redirect expected result
boosting the queue to 30
Hm, I'm not sure if it's case sensitive or not
the two character ones do not seem to be
finished all of them, in any case
boosting queue to 60
Interestingly, all of these seem to redirect to a warning page on xdotvu.com
but the URL includes the real target, so it's good enough for our purposes
getting a few timeouts, but ... it's going away soon. So, queue up to 90
we seem to have about 90 total warriors right now; so I'll boost the queue to 100, that way everybody can join in
[04:16]
Finished the initial-digit-three-character ones.
Hm, bunch of errors, dropping queue down to 90
[04:42]
...... (idle for 27mn)
checked over 100,000; nearly 4,000 found. [05:09]
........ (idle for 37mn)
cleared out the errors, now reloading the queue with 60 [05:46]
errors came back; draining queue, then will try 40 [05:54]
40 seems to work, trying 50 [06:02]
............................ (idle for 2h18mn)
***dashcloud has quit IRC (Read error: Operation timed out)
dashcloud has joined #urlteam
[08:20]
.................. (idle for 1h28mn)
JAASomebody2: As far as I can tell, the paid short (less than 3 characters) codes are case-insensitive, but the automatic six-character chodes are case-sensitive. No idea about the ones you can set yourself in the advanced options. [09:48]
..... (idle for 22mn)
"terroroftinytown.client.errors.ScraperError: Number of attempts exceeded for 5708844300 (0-DHxu)." Hmm. [10:10]
***T31M has joined #urlteam
T31M has quit IRC (Leaving)
[10:18]
.............................................. (idle for 3h45mn)
zhongfu_ has joined #urlteam
zhongfu has quit IRC (Ping timeout: 260 seconds)
[14:05]
............... (idle for 1h14mn)
Jonison has joined #urlteam [15:19]
Jonison has quit IRC (Ping timeout: 260 seconds) [15:27]
Somebody2I turned *off* gg-gg...
x-vu has finished the 3-character ones
and it looks like go-usa-gov has blocked us
I'm trying to drain the queue on it, then we'll leave it off for a few days
[15:40]
astridassholes
:P
[15:47]
Somebody2Eh, go-usa-gov is fine; they politely came in a day or so ago, and asked about setting up a bulk export instead.
I just thought I'd keep the scraper running till they actually told us to stop.
[15:47]
astridyeah [15:48]
Somebody2eh, it doesn't seem to be draining; I'll just reset the autoqueue back to the last result, and clear it
that way it won't interfere with the other jobs
ok, going afk for the day
[15:51]
astridtoodleoo [15:52]
....................................................... (idle for 4h34mn)
***svchfoo1 has quit IRC (Remote host closed the connection)
svchfoo3 has quit IRC (Remote host closed the connection)
svchfoo3 has joined #urlteam
svchfoo1 has joined #urlteam
JAA sets mode: +o svchfoo1
svchfoo1 sets mode: +o svchfoo3
[20:26]
astrid sets mode: +ooo joepie91_ HCross2 HCross [20:39]
svchfoo3 has quit IRC (Remote host closed the connection)
Aoede has left WeeChat 1.9
svchfoo3 has joined #urlteam
svchfoo1 sets mode: +o svchfoo3
[20:52]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)