[00:13] wait there are people trying to sell this book? [00:13] http://www.amazon.com/gp/product/0966288912 [00:16] http://contemporary-home-computing.org/1tb/wp-content/uploads/screenshot_200.png [00:16] rofl [00:16] nsfw [00:20] hahaha [00:20] i'm going to buy that book [01:02] Hmm, ndurner isn't here and I haven't managed to get anything from groups in hours. Suboptimal. [01:19] Wyatt: ditto. My discovery crawler has been going well, but the downloaders have all been idle for 20-ish hours [01:20] https://skitch.com/scumola/fu7nu/cacti [02:05] DFJustin: That's intriguing [02:05] I can do that now with the hadoop infrastructure, but not publicly (yet, at least) [02:06] It's very resource intensive [03:02] hot damn [03:02] it actually fucking worked [03:02] I don't know who requested it [03:02] but http://tracker.archive.org/torrent/torrents/FRIENDSTER-000000000-all.torrent is valid and everything [03:02] That was a good stress test [04:36] http://timemachine.6x.to/ [04:40] I'm so happy, these groups fo hams have been FLOODING me with descs. [04:41] A 268 issue run may be totally described in no time at all. [06:07] underscor: also is there a reason why archive.org labels all AVI files as Cinepack [06:07] He's even less qualified to answer these questions than me. [06:07] But AVI is a container format, which they called Cinepack. [06:07] The video handling is ALL based off of ffmpeg, if that helps at all. [08:43] underscor: Sorry, couldn't restrain myself, had to try the Friendster torrent again. I hope it didn't cause too much trouble. Your new torrent thing works wonderfully. [09:01] Well, it only kills MY machine [09:07] Oh, that's not a problem then. :) [09:18] Well, technically you are rsyncing to a machine crippled by the torrent generation. [09:18] So it's your problem too. [09:18] They gave him a new machine to destroy, so the trouble will pass momentarily. [16:25] what are you archiving exactely? [16:31] EVERYTHING [16:33] okay [16:35] archive all the HP Touchpad free apps before HP takes that site offline... [16:37] good idea [16:38] i'm archiving the stanford engineering everywhere courses [16:39] the torrent links there are already dead so i'm having to get them from youtube [17:04] SketchCow: It's off your machine now, has been for 2 days :P [17:05] alard: Does it seem like a valid torrent/downloads okay? [17:33] Oh thank good [17:33] ness [17:35] heh [17:40] 47.9GB 106:58:36 [ 130kB/s] [17:40] Total bytes written: 51438039040 (48GiB, 131KiB/s) [17:40] very slow [17:42] http://www.youtube.com/watch?v=5yrA-BpWECI&feature=player_embedded [17:46] underscor: Yes, my Deluge torrent client accepted it and started downloading from the web seed. I didn't wait for it to finish, of course, but everything seemed ok. [19:45] I've failed over a dozen times to download http://www.archive.org/download/FRIENDSTER-000000000/friendster.000023000-000240000.tar.gz [19:55] db48xOthe: Via torrent or http? [20:20] db48xOthe: I have just downloaded that file, the sha1 checksum matches the one in the archive.org _files.xml [20:20] But it seems that there is a problem with the gzip format of the file. [20:21] gzip: stdin: invalid compressed data--format violated [20:22] after 1,63 GB. [20:36] uh oh [20:38] hmmm. [20:40] According to the wiki, I uploaded that file. I don't have a copy, so I can't check where it went wrong. [20:54] alard: hahaha, I love the 193M log file the friendster download generated [20:54] http://tracker.archive.org/torrent/logs/ [20:59] ndurner: What is error 444 again? [20:59] Connecting to archiveteamorg.appspot.com|74.125.53.141|:80... connected. [20:59] HTTP request sent, awaiting response... 509 Bandwidth Limit Exceeded [20:59] who's in charge of this? we should consider self-hosting in future. [21:00] chronomex: It's not really bw limit exceeded iirc [21:00] oh? [21:00] It means there's no work or something like that [21:00] underscor: Ah, yes, I noticed that. :) -- db48x's wants to solve that, but no-one listens: https://savannah.gnu.org/bugs/index.php?33654 [21:00] oh. damn. [21:00] You have to ask ndurner [21:00] ok. [21:00] 444 = nothing to do ATM [21:00] * chronomex runs discover process then [21:01] The groups thing seems to be bandwith-limited a lot, these days. [21:01] 509 = throttled so we're not overshooting the quota [21:02] after ggroups pulls down their files permanently, we should use this list of group names to suck in messages too. [21:02] ndurner: There's no way to increase the quota? [21:02] (How do they track quota anyway?) [21:02] Er, rather, what do they limit? [21:03] other than paying up, no [21:03] requests per hour I think [21:03] oh god no not the $12 fee [21:03] the quota is about using or blocking resources [21:03] aren't there opensource GAE-compatible environments available? [21:04] (Thread.sleep() doesn't actually *use* CPU, but blocks it, so you still pay) [21:04] huh [21:04] ndurner: How much would it cost to open it up? [21:05] http://code.google.com/appengine/docs/billing.html [21:06] hard to say, because there are no real limits [21:06] You can set a "Max Daily Budget" though, can't you? [21:06] yes [21:06] yes [21:07] Oh, I see what you're saying [21:07] Each additional scraper/downloader increases the resource requirements [21:09] Jason's doing a 'hangout' on G+ if you guys are interested. I'm on there now too. [21:09] 1) link? 2) how is that different from irc exactly? [21:09] Video, iirc? [21:10] muelti-person video. [21:10] oh, video [21:10] I don't have any video gear, strangely enough [21:10] http://ge.tt/#8kWBpB7 [21:10] Friend just finished mixing his cover of Skyscraper <3 [21:12] I confuse huddle and hangout a lot [21:15] if anyone cares, this is me: https://plus.google.com/u/2/118060174030033503719/ [21:22] chronomex: Is that your real name? [21:22] my legal name is "Duncan Smith"; that is a Hangul transliteration of my legal name. [21:23] Oh, I see [21:23] I do not normally look like a vintage tugboat [21:24] lol [21:24] I don't approve of the conflation of "legal name" with "true name" [21:24] swit deonkeon, eh [21:24] chronomex: You're a boat? :) [22:20] aww [22:20] my Friendster archive topped out at 995GB [22:28] 10^12 bytes is 931 gibibytes, so using hd manufacturer logic you could call it over a terabyte :) [22:28] heh [22:29] * db48x2 throws a --si on there [22:29] 1.1T total [22:29] :) [23:12] Damn! [23:32] Adding cmalp_00001 to the torrent generation queue [23:32] Recieved request from 127.0.0.1 for cmalp_00001 - 2011-08-26 23:31:36.966885 [23:32] Satisfied request from 127.0.0.1 for cmalp_00001 - 2011-08-26 23:31:36.969984 [23:32] <3 [23:32] I love it [23:34] Nice and fast, too [23:34] underscor: looks like you've done a good job on that [23:42] Any other people need slots? [23:48] db48x2: Aranje Thanks <3 [23:48] <3