| Time |
Nickname |
Message |
|
12:14
🔗
|
ivan` |
underscor: during the upcoming greader grab, would it be feasible to run queries over all the warc.gz's uploaded to the server to find more URLs? |
|
12:20
🔗
|
Smiley |
ivan`: we've had warriors report new usernames before. |
|
12:20
🔗
|
Smiley |
No reaason why we can't report new urls. |
|
12:20
🔗
|
Smiley |
*that I know of* |
|
12:20
🔗
|
Smiley |
ersi: check it's not 0bytes etc. |
|
12:20
🔗
|
Smiley |
just some really basic validation. |
|
12:20
🔗
|
ivan` |
I was thinking about putting this into the greader-grab pipeline, but it's going to delay things by a day or two |
|
12:20
🔗
|
ivan` |
and I don't really know which URLs I want yet |
|
12:20
🔗
|
ivan` |
I know some, but not all |
|
12:23
🔗
|
ersi |
Smiley: Uh, yeah - if you don't remember: You wrote the exact same line yesterday evening. |
|
12:29
🔗
|
Smiley |
yeah I rmemeber |
|
12:29
🔗
|
Smiley |
:D |