[00:51] dashcloud: http://www.webcitation.org/ [00:58] fascinating- but considering I'm not (nor do I expect to be) an author of a scholarly work, it doesn't seem to be directed towards mass-market works (which is all I have)- if I'm wrong, please tell me [02:18] I will aim back at hackercon videos down the line [07:11] http://ask.slashdot.org/story/11/07/20/0121248/Ask-Slashdot-Best-Offline-Storage-Method-For-Large-Archives [07:36] Interesting, so tape is still king, and they still haven't figured out how to make hard drives that still spin up after being unplugged for a long time. [07:48] I wonder about the longevity of SSD storage [07:48] how long do they retain data if powered off? [07:59] Ymgve: it's the wine corking problem [07:59] you wont know until you try, but do you really want to risk valuable data on what could be a failed experiment? [08:00] Of course not, that's why you have a test data set [08:00] dur :) [08:01] then again, you can duplicate data [08:01] you can't duplicate wine :) [10:45] I'll leave this here [10:45] http://thepiratebay.org/torrent/6552595 [10:48] i recently thought about that [10:49] oops, typo in the title [10:49] here in germany public news stations have to "de-publicise" their online offerings after a while [10:49] which sucks [10:49] mirroing it all would be too much for individuals, especially if you include audio/video [10:49] ah [10:49] so a distributed redundant public archive of publically funded media would rock [10:50] oh and edit is disabled >_> [10:50] hmmm, that would be an interesting idea actually [10:51] the institutions would have to fight it though, since a lot of their content is copyrighted by third (non-public/-government) sources :( [11:24] Trying to download that torrent. [11:25] It's not coming down... yet. [11:41] 1738.2 / 4175.8 MB Rate: 145.2 / 5985.5 KB Uploaded: 16.9 MB [41%] 0d 0:06 [ R: 0.01] [11:41] News of the World Archive [11:41] There we go [12:04] # disallow archiving site [12:04] User-agent: ia_archiver [12:04] Disallow: / [12:04] what a douche [12:06] Who's that? [12:09] It's http://www.telecomramblings.com [12:10] SketchCow: want me to help get it? [12:14] The news of the world? or something else. [12:14] Or telecomramblings. [12:15] * SketchCow is doing 5 things this morning, ahving an awesome time doing it. [12:16] Today is the day I begin bringing ISOs to archive.org's Shareware CD collection. [12:16] Among other stuff. [12:16] news of the world [12:17] Neat [12:17] Well, it looks like SketchCow got NoTW bittorrent download handled.. Seeing how it's going at 6MB/a [12:17] /s* [12:17] Braaaaaaaaaaap [12:17] no, that is uploading 6mb/s [12:17] only coming down at 145k/s [12:18] Are you sure? Look at the ETA [12:18] I show it'll arrive in 4 hours. [12:18] In 4-5 hours it will be 100% seeded [12:18] Yeah. [12:19] I'll download it, get it up to archive.org, etc. [12:19] I'm also downloading all of that RantMedia show. [12:19] link to the torrent? [12:19] how big is it? [12:19] Today's tasks, I'm moving a lot of files. [12:19] Which torrent, news of the world of rantmedia. [12:19] or rantmedia. [12:19] notw [12:19] perfinion: 12:45 < Cameron_D> http://thepiratebay.org/torrent/6552595 [12:20] 91% 69GB 55.7MB/s 02:05 ETA [12:20] lol'd @ "news of the world of rantmedia" [12:20] See, 55MB/s. That's what I'm talking about. [12:20] MiniBits [12:21] this is more traffic on this channel than ive seen in days, whats up? [12:21] Me. [12:22] We all flock out of the lurkness hwn Sketchy is active. ;-) [12:22] I've been busy but I am doing archive team full time and am working on a bunch of stuff with getting our previous projects to resting homes. [12:22] I'm here in Kansas City, MO for the world's biggest Apple II only festival. [12:22] SketchCow: torrenting on a gigabit pipe? not fair [12:23] wow, my friendster data is still compressing, pushing 2 days now [12:23] Yes. [12:23] it'll take foreverrrrrrrr [12:23] yeah goddamn friendster had billions of files [12:24] I'm going to put the first million friends of friendster up, then will start backfilling our lists to figure out what exactly we got. [12:24] In fact... if anyone's feeling like it, could you go to the wiki and calculate how much we think we downloaded? [12:24] Out of the original, what is it, 124 million. [12:25] o.O none of the trackers on the notw tracker are working [12:25] perfinion: Patience [12:26] ersi: yeah it'll hit dht eventually [12:26] Right. [12:26] You'll hit the trackers, eventually [12:26] And then there's some asshole with a nice pipe. [12:26] P.S. It's in testing, but archive.org's moving into torrents. [12:26] I don't know the parameters of the effort, but it's being worked on. [12:26] torrents for distributing content out? [12:27] Yeah, another option. [12:27] the problem with torrents is you basically have to grab all or nothing [12:27] if its on a web mirror you can jsut browse the pars you want [12:36] from the wiki, here is what I added up as being left in the avaialble pool, http://i.imgur.com/V2h6c.png [12:36] http://www.archive.org/details/cdbbsarchive if you want. [12:38] I want the opposite, Cameron. I want what we GOT. I assume we're not downloading anymore. [12:38] But if that's true, we got 21 million accounts, not much. [12:38] I mean, unless you look, then you see how fucking huge 21 million is. [12:39] it was easier that way [12:39] yeah, 21 million is a lot, but not compared to how much was on the site [12:42] yeah pulling from friendster was slow [12:42] there were a ton of requests for not much stuff [12:43] http://www.archive.org/details/WILDCATGOLD&reCache=1 is one of the examples of me re-doing this CD Shareware collection. [12:43] Basically, they got a stack of CDs from this group. [12:43] And to be frank, the integration job was shit in a box. [12:44] But it appears we got the first 14 million friendster accounts, then. [12:44] That's critical. [12:49] We downloaded a total of 20,438,245 [12:49] profiles [12:50] OK. [12:50] Thanks for that info. [12:50] While it's sad we lost a lot, we have enough for future generations to see what friendster was about. [12:55] http://www.archive.org/details/cdbbsarchive improved the description. [13:00] OK, gotta head upstairs, but I'll be back throughout the day to set more processes going, etc. [13:18] SketchCow: do you have a kryoflux [13:18] i'm seriously considering picking one up in time for the CCC conference in August [13:18] I suppose a lot of people there might have something worth storing [13:19] Bought one, it'll be waiting for me when I get back. [13:19] Ok [13:19] Let me know if it is as good as they say [13:20] I got a catweasel, fragile beasts they are [13:20] I don't understand how it can be completely independent of the disk drive you use [13:20] too bad c64 reading needs a drive mod and I'm too lazy to do it [13:20] KryoFlux claims they can read c64 without hardware mods [13:20] jch: old PC floppy drives are basically pretty dumb [13:21] jch: weird [13:21] Cat Weasel: Terrible [13:21] Kryoflux: Finally great [13:21] It's just weird to me that everything goes on in software, w.r.t. reading stuff of disks [13:21] FC5025: Excellent, now obsoleted by kryoflux [13:22] I'll respond to this window a little later today, talk later folks [13:22] jch: i'm pretty sure you still need a mod for reading c64 disks, at least double sided ones [13:23] unless you're lucky with the drive you got [13:23] I wonder. [13:23] (I'm too young to have any real knowledge on 5.25 inch floppies, let alone 8 inch ones) [13:24] c64 is special in that it only has one read head, in contrast with a lot of other drives who have to heads [13:25] two [13:25] dude.. the 'mod' is to flip the disk over like everyone else had to do :) [13:26] Cowering: no, because a lot of PC drives NEED to detect sector 0 - the little hole near the big one in the center of the disc [13:27] oh.. true.. stupid PCs [13:27] with the disk the wrong way around, the PC drive doesn't sense anything since the IR is blocked, and won't work [13:27] the dual drive amiga reading hack for PC should work for that though [13:27] you also can't use the fact that the PC has two read heads because the PC read heads are offset like 3-5 tracks away from each other [13:28] anyone have the link to the rantmedia torent [13:32] http://www.siliconsonic.de/t/flipside.html [13:33] SketchCow: yeah, I'm just not much of a hardware mod guy [13:33] anyway, for non-original disks, a plain serial 1541 works wonders still [13:34] http://forum.kryoflux.com/viewtopic.php?f=3&t=25&p=1589&hilit=c64+flippy#p1589 is the discussion. [13:35] Bear in mind, my tollerance for finger wagging geek-superior discussions is about 0. [13:35] +/- 0. [13:35] I just want to get shit done. [13:35] I went downstairs and realized with the time shift I'm an hour early. [13:37] So here I am again, making the world better. [13:37] Just realized I need to s3 something in, damnit [13:44] Wow. [13:44] OK, so I never run the --delete option on bitsavers [13:44] So I did notice I was seeing double things, i.e. file.pdf in both /apple and /apple/apple_ii [13:45] so I thought, oh, they must be removing and moving doubles [13:45] So I do an rsync with --delete [13:45] Dry run, of course. [13:45] They actually deleted a bunch of digitized books, too [13:45] Guess someone got to them. [13:46] So now I have to write this wasteful script [13:46] That will do a find against all deleted items [13:46] if nothing with that name is there, put it aside [13:47] this historian does not approve [13:49] I don't like that I can't trust the thing I'm mirroring to do something acceptable [13:56] Al does not ever delete things.. i bet all the files are on some other folder now... [14:01] I am sure they're on a folder, but not on bitsavers. [14:01] I just wrote a program. [14:02] 8078378 19072 -r--r--r-- 1 6018 6018 9742343 Sep 22 2004 ./pdf/univOfWiscMilwaukee/UWMCC_1401_OSver2_Jul68.pdf [14:02] Original file is in pdf/uwMilwaukee/UWMCC_1401_OSver2_Jul68.pdf [14:02] 8078387 19072 -rwxrwxrwx 1 root wheel 9742343 Sep 22 2004 ./pdf/uwMilwaukee/UWMCC_1401_OSver2_Jul68.pdf [14:03] See, in that case, the file went from one directory to a another. Fine. [14:03] but I'm finding ones where no, in fact it's MIA [14:05] Not many, mind you, but enough I have to now run this massive findfest [14:09] Example: [14:09] HP1000-21XX_CHM_Source_Code_License_final.pdf.... [14:09] 7067255 216 -rwxrwxrwx 1 root wheel 110350 Jan 22 2008 ./bits/HP/HP1000-21XX_CHM_Source_Code_License_final.pdf [14:09] Original file is in bits/HP/HP1000-21XX_CHM_Source_Code_License_final.pdf [14:12] i got a 2G of these now http://www.kimsufi.co.uk/ [14:15] i also got a working setup to download files from http://atomicgamer.com/ automated from the shell now, if anyone else is interested (yeah right) [14:15] Anybody from the AT going to the CCC Camp in Berlin in three weeks [14:16] i'd love to [14:16] It would be ideal for getting shit done [14:17] We might even get some new members [14:17] yeah, AT would make a nice lightning talk topic [14:18] http://events.ccc.de/camp/2011/Fahrplan/ [14:23] rsync has a --backup option that could be useful [14:24] also you may be able to search through http://bitsavers.org/pdf/IndexByDate.txt rather than traversing the directory structure yourself [14:25] I ordered a kryoflux but I do have a lot of flippy disks so I will probably still need to mail them to someone [14:45] jch: I'm going [14:45] jch: You? [16:19] http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/ [16:38] i have a kryoflux on the way as well [16:39] and for 3.5", i have a working autoloader i can use. [17:55] Anyone have an idea if IA's liveweb/wayback thing gobbles up other material than just the site? Like pdf's and such? [18:36] Mmmh, bittorrent is "funny".. I got 3 peers connected (one which is slurpin' hard, one who's sipping and one who's sleeping) and I can see 9 in total in the swarm - but they aren't connecting/my client isn't connecting to them [18:36] News of the World Archive [18:36] 4048.9 / 4175.8 MB Rate: 468.2 / 36.6 KB Uploaded: 9389.2 MB [96%] 0d 0:59 [ R: 2.32] [18:39] Wooho, you're getting there [19:25] SketchCow: http://echoprint.me/data_download <- something to insert into archive.org [19:25] s/insert/ingest [19:39] ersi: Where's the torrent link? [19:40] I'll make one for you if you want [19:40] if you don't, there's afaik only the HTTP download from there [19:40] SketchCow: USB harddisks are so 2008, esata is where its at [19:41] Spirit_: Sure, but there's not that many eS-ATA ports around [19:41] i have two on my 2 year old mainboard [19:41] I was more thinking about to plug in directly, as in chassis [19:42] that would be even easier, my mainboard is 4 internal. i would just need some whatchacallits [19:42] chassis, yes :D [19:42] has [19:42] god [19:42] ENGRISH! [19:42] wat [19:42] Case is proper? [19:43] that would be even easier, my mainboard has 4 internal sata ports. i would just need some chassis accessible from the outside [19:43] if that was what you meant [19:43] no, I meant a port in the chassi/case [19:46] something like http://p.gzhls.at/534865.jpg [19:46] No. [19:47] This -> http://1toppc.com/Merchant2/images/Notebook/Asus/M50/comboport.jpg [19:47] yeah i got 2 of those [19:47] :d [19:48] I don't [19:48] don't think I have one on a single 'puter I got ;p [19:48] damn [19:48] perhaps on one or two of the motherboards though [19:48] well, i chose my mainboard well [20:32] a [21:25] Oh boy, let's all jump to esata [21:25] because, you know, usb stopped working [21:25] poof [21:27] other day I saw a usb external drive that didn't have a converter in it [21:27] damn thing had a usb jack right on the disk's ckt board [21:27] first time I've seen that [21:30] eSATA harddisks are so 2010, Thunderbolt is where it's at [22:04] A single 15k rpm HD can only output 1.5Gbps though (USB 1.0 speed), so what's the point with a single HD and anything > USB 2.0? I could see the need for a RAID or SSD or something, but for a single HD, what's the point? [22:05] oops, 1.5Gbps is USB 2.0. My bad. [22:13] Yeah, usb 1.0 was what, 11mbit? [22:13] uh [22:13] usb2 is 480Mbps [22:14] NOT FAST ENOUGH [22:18] USB 1.0 was only fast enough to run an external CD drive at about 4x max, IIRC [22:18] tev?!? [22:18] how long have you been in here? [22:19] I can't remember off the top of my head, probably a week or two [22:20] crazy times [22:20] I had actually been here quite a while back too when I first saw the website, but just briefly [22:20] aye [22:22] did you guys see the new Backblaze storage pods? [22:24] http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/ [22:32] interesting [22:33] cheaper & better than the 2009 model [22:33] oh but of course :P [22:34] as an addition to that, the company that makes the cases will also sell a fully-assembled model (no hard drives included) for $5500 or so [22:45] just under 57k for 1000 TB worth of storage (and another 50k or so to power/etc the beast) [23:02] protocase is selling full systems? I thought they were only selling empty cases [23:03] just the cases [23:03] * atg wishes he had someway of justifying all that storage [23:03] but apparently some of the parts are tricky to buy in small quantities, or require you to visit some shady places [23:03] On the plus side, I didn't realize that SATA port expanders were available as cheaply as the are [23:03] (by full systems I mean with the electronics and backplanes and stuff, just no hard drives) [23:03] Dunno how good they are though... :| [23:04] atg: apparently they only work well if your controller uses SiI controller chipsets [23:04] are SiI chipsets better able to deal with long cables? [23:05] (based on their 2009 installment about the pods) [23:05] chronomex: it had to do with SiI being the originator of the port expansion part of the spec and compatibility issues with other chips [23:06] ah [23:06] http://blog.backblaze.com/â2009/10/12/user-builds-extâreme-media-server-based-onâ-a-backblaze-storage-pod/ [23:06] that blog is from someone who actually built the original 2009 model [23:06] a user like me or you [23:06] there are comments from me on that blog [23:07] (the blogspot one) [23:07] atg: just spin the consolidation angle, and possibly power-savings [23:08] you do have a mini-server farm someone in your residence right? [23:08] Well yes, but currently putting monnies into networking [23:08] friend of mine is doing the storage :p [23:08] networking and VMs [23:09] if you wait a bit longer, Intel's supposed to be rolling out 10 Gig Ethernet desktop cards next year (single chip, not huge power hogs) [23:09] My networking stuff is all gigabit at the moment... I can do aggregrate links which will be fine for just about everything [23:10] dual gigabit links agg'd together should be fine for media use / VMs/etc [23:10] (yay older dell gigabit switches and ebay) [23:11] link aggregation will not get you a single stream over 1Gbit [23:14] Hmm. I hadn't realized that [23:15] the total of all streams can go over, but no single stream will be able to go over [23:16] Ah [23:16] and, iirc, it generally uses the mac addresses to keep track of which port a stream is on, and changes it periodically [23:16] Still, for almost everything I can envision, an agg'd link from the fileserver to the core switch should be fine for everything else accessing it [23:16] (it's been a couple years since I played with it) [23:16] Seeing as everything else will just have a single link, it shouldn't be an issue anyways