[12:36] we are snails [12:37] man, wikis are being desstroyed anywhere [12:37] go go go [12:37] who have good bandwidth to donwload a fuckton of wikis? [13:03] Need help with the tracker, or are you also looking for tracker hosting? [13:03] mm [13:03] i think we can use a simple method [13:04] a script that reading a list file, launch the downloader [13:04] tracker is a bit advanced [13:05] Yes, simple is good. Although a tracker may be useful if you're often updating the lists. [13:20] calculating a random sample from a 20,000 wikis list, about 50% are dead [13:20] list is from 2009 [13:21] im going to discard those ones, and split the result list in 100 wikis batches [13:22] and profit [14:32] generating lists... [14:32] http://code.google.com/p/wikiteam/wiki/TaskForce [15:16] lists generated http://code.google.com/p/wikiteam/source/browse/trunk#trunk%2Fbatchdownload%2Flists [19:38] alard, we could use some help with coding [19:38] What kind of help? [19:39] alard,https://code.google.com/p/wikiteam/issues/list?can=2&q=&sort=priority&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary [19:40] emijrp created http://wikiteam.googlecode.com/svn/trunk/batchdownload/launcher.py [19:40] but I suspect it doesn't have any sort of error handling and so on [19:44] Ah, I see. I mostly responded to emijrp's question about trackers, though. I'm not that familiar with wiki software. [19:55] Yes, some sort of tracker will be needed later... [20:44] man, the first wiki in the list000 has a lot of pages [20:45] i hope others are small [20:45] and it is hosted in an IP [20:45] first list contain a lot of IP domains [21:09] emijrp, the problem is that as usual there are lots of errors [21:09] and it's hard to keep track of them, the script just goes on and you have to scroll back [21:09] save the output [21:10] ? [21:12] you can redirect the console text to a file [21:12] 2> a.txt [21:15] what errors do you get? [21:15] all urls where alive [21:15] were* [22:12] Should we submit our logs somewhere? [22:12] I've had a few different errors [22:12] Also, alard dcmorton ersi ops spread por favor [23:00] underscor, what sort of errors? [23:00] They're just the usual errors for me, already in the issue tracker.