[07:01] <Nemo_bis> oh no! the upload of the 50+ GiB archive has started before I had time to fix the curl line to give me progress info
[07:01] <Nemo_bis> On the other hand, speed seems reasonable now
[07:02] <Nemo_bis> http://archive.org/details/wiki-halowikinet took three times what my bandwidth would have allowed
[07:03] <underscor> Downloaded 50030 pages :o
[07:03] <Nemo_bis> Hm, so little?
[07:03] <underscor> :P
[07:03] <underscor> that's the biggest one I've done so far
[07:03] <Nemo_bis> Seriously, I'm at several dozens millions
[07:04] <Nemo_bis> ah ok, a single wiki
[07:04] <Nemo_bis> My biggest one is a million and a half I think
[07:04] <underscor> wow
[07:04] <underscor> that's cool
[07:14] <Nemo_bis> metacafe is 7 https://wikistats.wmflabs.org/display.php?t=mw&s=total_desc
[07:15] <Nemo_bis> underscor: it would be wonderful if you could try werelate, it doesn't have an API so parsing HTML list takes ages
[07:15] <Nemo_bis> and I mean weeks on an average server
[07:16] <underscor> oh wow
[07:16] <underscor> what incantation do I need to run?
[07:19] <Nemo_bis> hm?
[07:19] <Nemo_bis> just dumpgenerator.py on it index.php
[07:21] <underscor> I didn't know if there were any args I'm supposed to pass
[07:21] <underscor> I haven'r run dumpgenerator in like 7+ months
[07:21] <underscor> haven't*
[07:25] <Nemo_bis> underscor: nothing special, just the usual --index=URL --xml --images
[09:08] <Nemo_bis> hmm the log check is producing some more dumps http://p.defau.lt/?_8hfXraVny3wzx_HYzG9Wg
[10:41] <Nemo_bis> emijrp: now we could start using http://archive.org/search.php?query=originalurl%3Astupidedia
[10:41] <Nemo_bis> wikistats.wmflabs.org could link to the dumps via an originalurl: search of the api url
[10:42] <Nemo_bis> and maybe even fetch the last-updated-date
[12:37] <Schbirid> http://dpwiki.slipgateconstruct.com/ might shut down, http://dpwiki.slipgateconstruct.com/api.php
[16:57] <underscor> dumping
[18:40] <Schbirid> thanks
[20:25] <underscor> ugh, very slow :(
[20:25] <underscor> weird, only 32 pages?
[20:27] <Schbirid> heh, might be, actually
[20:57] <Nemo_bis> Special:Statistics