| Time |
Nickname |
Message |
|
07:01
🔗
|
Nemo_bis |
oh no! the upload of the 50+ GiB archive has started before I had time to fix the curl line to give me progress info |
|
07:01
🔗
|
Nemo_bis |
On the other hand, speed seems reasonable now |
|
07:02
🔗
|
Nemo_bis |
http://archive.org/details/wiki-halowikinet took three times what my bandwidth would have allowed |
|
07:03
🔗
|
underscor |
Downloaded 50030 pages :o |
|
07:03
🔗
|
Nemo_bis |
Hm, so little? |
|
07:03
🔗
|
underscor |
:P |
|
07:03
🔗
|
underscor |
that's the biggest one I've done so far |
|
07:03
🔗
|
Nemo_bis |
Seriously, I'm at several dozens millions |
|
07:04
🔗
|
Nemo_bis |
ah ok, a single wiki |
|
07:04
🔗
|
Nemo_bis |
My biggest one is a million and a half I think |
|
07:04
🔗
|
underscor |
wow |
|
07:04
🔗
|
underscor |
that's cool |
|
07:14
🔗
|
Nemo_bis |
metacafe is 7 https://wikistats.wmflabs.org/display.php?t=mw&s=total_desc |
|
07:15
🔗
|
Nemo_bis |
underscor: it would be wonderful if you could try werelate, it doesn't have an API so parsing HTML list takes ages |
|
07:15
🔗
|
Nemo_bis |
and I mean weeks on an average server |
|
07:16
🔗
|
underscor |
oh wow |
|
07:16
🔗
|
underscor |
what incantation do I need to run? |
|
07:19
🔗
|
Nemo_bis |
hm? |
|
07:19
🔗
|
Nemo_bis |
just dumpgenerator.py on it index.php |
|
07:21
🔗
|
underscor |
I didn't know if there were any args I'm supposed to pass |
|
07:21
🔗
|
underscor |
I haven'r run dumpgenerator in like 7+ months |
|
07:21
🔗
|
underscor |
haven't* |
|
07:25
🔗
|
Nemo_bis |
underscor: nothing special, just the usual --index=URL --xml --images |
|
09:08
🔗
|
Nemo_bis |
hmm the log check is producing some more dumps http://p.defau.lt/?_8hfXraVny3wzx_HYzG9Wg |
|
10:41
🔗
|
Nemo_bis |
emijrp: now we could start using http://archive.org/search.php?query=originalurl%3Astupidedia |
|
10:41
🔗
|
Nemo_bis |
wikistats.wmflabs.org could link to the dumps via an originalurl: search of the api url |
|
10:42
🔗
|
Nemo_bis |
and maybe even fetch the last-updated-date |
|
12:37
🔗
|
Schbirid |
http://dpwiki.slipgateconstruct.com/ might shut down, http://dpwiki.slipgateconstruct.com/api.php |
|
16:57
🔗
|
underscor |
dumping |
|
18:40
🔗
|
Schbirid |
thanks |
|
20:25
🔗
|
underscor |
ugh, very slow :( |
|
20:25
🔗
|
underscor |
weird, only 32 pages? |
|
20:27
🔗
|
Schbirid |
heh, might be, actually |
|
20:57
🔗
|
Nemo_bis |
Special:Statistics |