| Time |
Nickname |
Message |
|
07:29
🔗
|
* |
Nemo_bis still downloading shoutwiki.com at 5 KB/s max :-( |
|
12:07
🔗
|
Nemo_bis |
OMG, http://icehockey.shoutwiki.com has 80 000 pages |
|
12:31
🔗
|
ersi |
sounds like spam |
|
16:10
🔗
|
LobStoR |
80,000 just for ice hockey? that is madness. |
|
16:12
🔗
|
Nemo_bis |
indeed |
|
16:13
🔗
|
Nemo_bis |
I had to split the task, now I have an instance only for that wiki |
|
20:39
🔗
|
emijrp |
english wikitravel finished, but no images |
|
20:47
🔗
|
emijrp |
by the way, most images are on this shared project http://wikitravel.org/shared/Main_Page |
|
20:48
🔗
|
Nemo_bis |
ah, nice |
|
20:48
🔗
|
Nemo_bis |
why can't they just use Commons? |
|
20:48
🔗
|
Nemo_bis |
Do they have many unfree images? |
|
20:49
🔗
|
emijrp |
maybe |
|
20:53
🔗
|
LobStoR |
is it allowable within Commons' policy to upload images that are *intended* for use outside of Wikimedia Foundation projects? |
|
20:54
🔗
|
emijrp |
i dont think so |
|
20:54
🔗
|
emijrp |
but if they are on Commons scope, maybe |
|
20:57
🔗
|
emijrp |
We are glad to announce the 100th wiki dump of WikiTeam: English WikiTravel: http://code.google.com/p/wikiteam/downloads/detail?name=wikitravelorg_wiki_en-20110605-current.xml.7z |
|
21:01
🔗
|
LobStoR1 |
i am running a script to automatically generate a torrent for the upcoming enwiki-20110722-pages-articles.xml.bz2 |
|
21:03
🔗
|
LobStoR1 |
1) it checks the RSS on dumps.wikimedia.org every 60 seconds, 2) as soon as the new file is published, 3) it feeds the http link to burnbit.com, which generates a torrent |
|
21:04
🔗
|
LobStoR1 |
i'm using Yahoo Pipes to generate a search-based RSS feed from burnbit.com, which i have plugged into my torrent client for automatic downloads |
|
21:05
🔗
|
emijrp |
why don't you use the 7z one? |
|
21:05
🔗
|
LobStoR |
pages-articles is only in bz2 |
|
21:05
🔗
|
emijrp |
ah, ok, page-articles |
|
21:05
🔗
|
LobStoR |
pages-meta-history is in bz2 or 7z |
|
21:05
🔗
|
emijrp |
average size? |
|
21:05
🔗
|
LobStoR |
here's a link to the Pipe, if you're interested http://pipes.yahoo.com/pipes/pipe.run?_id=f55afdb4a562a2a7ec5096624ac9a4e6&_render=rss&q=enwiki+pages-articles |
|
21:06
🔗
|
LobStoR |
i assume the next enwiki pages-articles will be slightly larger than the last, which was 6.92 GB |
|
21:06
🔗
|
emijrp |
interesting, can you do that with hhttp://code.google.com/p/wikiteam/downloads/list ? |
|
21:07
🔗
|
LobStoR |
yes, i'll have it done by tomorrow |
|
21:07
🔗
|
emijrp |
paste your links here http://archiveteam.org/index.php?title=Wikiteam |
|
21:07
🔗
|
LobStoR |
rgr |
|
21:08
🔗
|
LobStoR |
i've used Yahoo Pipes to take existing Podcast feeds (with http links), then automatically convert everything into a .torrent link using the Burnbit service |
|
21:09
🔗
|
Nemo_bis |
LobStoR, yes, it's certainly allowed. Commons' scope is pretty broad |
|
21:13
🔗
|
emijrp |
downloading the last page-articles torrent |
|
21:25
🔗
|
* |
Nemo_bis is using up all bandwidth for JSTOR torrent |
|
21:27
🔗
|
LobStoR |
torrent feeds would be good for distributing wikiteam/archiveteam data automatically |
|
21:27
🔗
|
LobStoR |
i'll definitely work on that |
|
21:37
🔗
|
emijrp |
i think internet archive could add a link to all their items to torrent, using ther server as webseed |
|
21:41
🔗
|
LobStoR |
hmmm, the pipe i made a few days ago is now broken, due to Burnbit adding a captcha |
|
21:41
🔗
|
LobStoR |
they must have thought it was some sort of bot |
|
21:41
🔗
|
LobStoR |
which, i guess, it sort of is |
|
22:12
🔗
|
LobStoR |
ok, emijrp... i still have some tweaks to do, but the wikiteam torrent feed is ready |
|
22:12
🔗
|
LobStoR |
i've tested successfully it in utorrent on windows (which downloads from the "enclosure" URL) http://pipes.yahoo.com/pipes/pipe.run?_id=2f15d291dd975362a2dfdb1da3c3ba4d&_render=rss&project=wikiteam |
|
22:15
🔗
|
LobStoR |
Google Code uses atom feeds, so i still need to adapt some of the elements to work properly on RSS (such as the dates), but it works without them, for now |
|
22:27
🔗
|
LobStoR |
"pipes.run" keeps triggering the spam blacklist on the wiki for ".ru" :-( |
|
22:30
🔗
|
Nemo_bis |
are all .ru domains blacklisted? where? |
|
22:35
🔗
|
LobStoR |
archiveteam.org |
|
22:35
🔗
|
LobStoR |
not Wikimedia |
|
22:35
🔗
|
LobStoR |
http://archiveteam.org/index.php?title=Wikiteam#BitTorrent_downloads |
|
22:35
🔗
|
Nemo_bis |
ah so not my fault :-p |
|
22:36
🔗
|
LobStoR |
haha, yes, one way of putting it |
|
22:36
🔗
|
Nemo_bis |
I can't manage to download those torrents |
|
22:36
🔗
|
Nemo_bis |
Are all torrent clients able to download from webseed? |
|
22:36
🔗
|
Nemo_bis |
eg Transmission |
|
22:36
🔗
|
LobStoR |
there's two different webseed standards |
|
22:36
🔗
|
LobStoR |
and not every client supports it |
|
22:36
🔗
|
LobStoR |
transmission doesn't support the webseed method on those torrent |
|
22:37
🔗
|
LobStoR |
give it a few minutes, and me and emijrp should be able to seed it to you ;-) |
|
22:38
🔗
|
Nemo_bis |
ok |
|
22:39
🔗
|
LobStoR |
it's a shame that there's two different webseeding standards, of which neither is really very well implemented http://en.wikipedia.org/wiki/BitTorrent_%28protocol%29#Web_seeding |
|
22:42
🔗
|
LobStoR |
perhaps we can coax the folks at Burnbit to support both webseed standards in their .torrent metafiles, so that they aren't alienating users... but for now, this is what we're stuck with |
|
22:56
🔗
|
Nemo_bis |
perhaps they're trying to make their standard more popular so that it becomes THE standard |
|
22:57
🔗
|
soultcer |
One webseed standard requires a cgi script to run on the webseed server |
|
23:01
🔗
|
LobStoR |
Yahoo Pipes has very aggressive server-side caching, which makes sense |
|
23:02
🔗
|
LobStoR |
but makes it difficult for me to confirm if i finally got the dates working properly, since i have to wait 15-30 minutes for the feed to reflect any changes to the Pipe |
|
23:02
🔗
|
Nemo_bis |
I wonder why they shut down Geocities but keep such a strange service |
|
23:08
🔗
|
LobStoR |
yeah, pipes probably really doesn't even bring them any revenue, since the whole point is to generate a feed... resulting in no pageviews on yahoo.com |
|
23:08
🔗
|
LobStoR |
unless they start inserting text ads into your feeds :-P |
|
23:18
🔗
|
LobStoR |
small filesizes (< 1 MB?) are blocked on Burnbit, which is unfortunately many of the Wikiteam downloads |