#archiveteam 2011-07-20,Wed

↑back Search

Time Nickname Message
00:51 πŸ”— bbot_ dashcloud: http://www.webcitation.org/
00:58 πŸ”— dashcloud fascinating- but considering I'm not (nor do I expect to be) an author of a scholarly work, it doesn't seem to be directed towards mass-market works (which is all I have)- if I'm wrong, please tell me
02:18 πŸ”— SketchCow I will aim back at hackercon videos down the line
07:11 πŸ”— ersi http://ask.slashdot.org/story/11/07/20/0121248/Ask-Slashdot-Best-Offline-Storage-Method-For-Large-Archives
07:36 πŸ”— Wyatt Interesting, so tape is still king, and they still haven't figured out how to make hard drives that still spin up after being unplugged for a long time.
07:48 πŸ”— Ymgve I wonder about the longevity of SSD storage
07:48 πŸ”— Ymgve how long do they retain data if powered off?
07:59 πŸ”— RedType Ymgve: it's the wine corking problem
07:59 πŸ”— RedType you wont know until you try, but do you really want to risk valuable data on what could be a failed experiment?
08:00 πŸ”— ersi Of course not, that's why you have a test data set
08:00 πŸ”— ersi dur :)
08:01 πŸ”— Ymgve then again, you can duplicate data
08:01 πŸ”— Ymgve you can't duplicate wine :)
10:45 πŸ”— Cameron_D I'll leave this here
10:45 πŸ”— Cameron_D http://thepiratebay.org/torrent/6552595
10:48 πŸ”— Spirit_ i recently thought about that
10:49 πŸ”— Cameron_D oops, typo in the title
10:49 πŸ”— Spirit_ here in germany public news stations have to "de-publicise" their online offerings after a while
10:49 πŸ”— Spirit_ which sucks
10:49 πŸ”— Spirit_ mirroing it all would be too much for individuals, especially if you include audio/video
10:49 πŸ”— Cameron_D ah
10:49 πŸ”— Spirit_ so a distributed redundant public archive of publically funded media would rock
10:50 πŸ”— Cameron_D oh and edit is disabled >_>
10:50 πŸ”— Cameron_D hmmm, that would be an interesting idea actually
10:51 πŸ”— Spirit_ the institutions would have to fight it though, since a lot of their content is copyrighted by third (non-public/-government) sources :(
11:24 πŸ”— SketchCow Trying to download that torrent.
11:25 πŸ”— SketchCow It's not coming down... yet.
11:41 πŸ”— SketchCow 1738.2 / 4175.8 MB Rate: 145.2 / 5985.5 KB Uploaded: 16.9 MB [41%] 0d 0:06 [ R: 0.01]
11:41 πŸ”— SketchCow News of the World Archive
11:41 πŸ”— SketchCow There we go
12:04 πŸ”— ersi # disallow archiving site
12:04 πŸ”— ersi User-agent: ia_archiver
12:04 πŸ”— ersi Disallow: /
12:04 πŸ”— ersi what a douche
12:06 πŸ”— SketchCow Who's that?
12:09 πŸ”— ersi It's http://www.telecomramblings.com
12:10 πŸ”— NovaKing SketchCow: want me to help get it?
12:14 πŸ”— SketchCow The news of the world? or something else.
12:14 πŸ”— SketchCow Or telecomramblings.
12:15 πŸ”— * SketchCow is doing 5 things this morning, ahving an awesome time doing it.
12:16 πŸ”— SketchCow Today is the day I begin bringing ISOs to archive.org's Shareware CD collection.
12:16 πŸ”— SketchCow Among other stuff.
12:16 πŸ”— NovaKing news of the world
12:17 πŸ”— ersi Neat
12:17 πŸ”— ersi Well, it looks like SketchCow got NoTW bittorrent download handled.. Seeing how it's going at 6MB/a
12:17 πŸ”— ersi /s*
12:17 πŸ”— SketchCow Braaaaaaaaaaap
12:17 πŸ”— NovaKing no, that is uploading 6mb/s
12:17 πŸ”— NovaKing only coming down at 145k/s
12:18 πŸ”— ersi Are you sure? Look at the ETA
12:18 πŸ”— SketchCow I show it'll arrive in 4 hours.
12:18 πŸ”— Cameron_D In 4-5 hours it will be 100% seeded
12:18 πŸ”— SketchCow Yeah.
12:19 πŸ”— SketchCow I'll download it, get it up to archive.org, etc.
12:19 πŸ”— SketchCow I'm also downloading all of that RantMedia show.
12:19 πŸ”— perfinion link to the torrent?
12:19 πŸ”— perfinion how big is it?
12:19 πŸ”— SketchCow Today's tasks, I'm moving a lot of files.
12:19 πŸ”— SketchCow Which torrent, news of the world of rantmedia.
12:19 πŸ”— SketchCow or rantmedia.
12:19 πŸ”— perfinion notw
12:19 πŸ”— ersi perfinion: 12:45 < Cameron_D> http://thepiratebay.org/torrent/6552595
12:20 πŸ”— SketchCow 91% 69GB 55.7MB/s 02:05 ETA
12:20 πŸ”— ersi lol'd @ "news of the world of rantmedia"
12:20 πŸ”— SketchCow See, 55MB/s. That's what I'm talking about.
12:20 πŸ”— Spirit_ MiniBits
12:21 πŸ”— bsmith093 this is more traffic on this channel than ive seen in days, whats up?
12:21 πŸ”— SketchCow Me.
12:22 πŸ”— ersi We all flock out of the lurkness hwn Sketchy is active. ;-)
12:22 πŸ”— SketchCow I've been busy but I am doing archive team full time and am working on a bunch of stuff with getting our previous projects to resting homes.
12:22 πŸ”— SketchCow I'm here in Kansas City, MO for the world's biggest Apple II only festival.
12:22 πŸ”— perfinion SketchCow: torrenting on a gigabit pipe? not fair
12:23 πŸ”— Cameron_D wow, my friendster data is still compressing, pushing 2 days now
12:23 πŸ”— SketchCow Yes.
12:23 πŸ”— SketchCow it'll take foreverrrrrrrr
12:23 πŸ”— perfinion yeah goddamn friendster had billions of files
12:24 πŸ”— SketchCow I'm going to put the first million friends of friendster up, then will start backfilling our lists to figure out what exactly we got.
12:24 πŸ”— SketchCow In fact... if anyone's feeling like it, could you go to the wiki and calculate how much we think we downloaded?
12:24 πŸ”— SketchCow Out of the original, what is it, 124 million.
12:25 πŸ”— perfinion o.O none of the trackers on the notw tracker are working
12:25 πŸ”— ersi perfinion: Patience
12:26 πŸ”— perfinion ersi: yeah it'll hit dht eventually
12:26 πŸ”— SketchCow Right.
12:26 πŸ”— ersi You'll hit the trackers, eventually
12:26 πŸ”— SketchCow And then there's some asshole with a nice pipe.
12:26 πŸ”— SketchCow P.S. It's in testing, but archive.org's moving into torrents.
12:26 πŸ”— SketchCow I don't know the parameters of the effort, but it's being worked on.
12:26 πŸ”— perfinion torrents for distributing content out?
12:27 πŸ”— SketchCow Yeah, another option.
12:27 πŸ”— perfinion the problem with torrents is you basically have to grab all or nothing
12:27 πŸ”— perfinion if its on a web mirror you can jsut browse the pars you want
12:36 πŸ”— Cameron_D from the wiki, here is what I added up as being left in the avaialble pool, http://i.imgur.com/V2h6c.png
12:36 πŸ”— SketchCow http://www.archive.org/details/cdbbsarchive if you want.
12:38 πŸ”— SketchCow I want the opposite, Cameron. I want what we GOT. I assume we're not downloading anymore.
12:38 πŸ”— SketchCow But if that's true, we got 21 million accounts, not much.
12:38 πŸ”— SketchCow I mean, unless you look, then you see how fucking huge 21 million is.
12:39 πŸ”— Cameron_D it was easier that way
12:39 πŸ”— Cameron_D yeah, 21 million is a lot, but not compared to how much was on the site
12:42 πŸ”— perfinion yeah pulling from friendster was slow
12:42 πŸ”— perfinion there were a ton of requests for not much stuff
12:43 πŸ”— SketchCow http://www.archive.org/details/WILDCATGOLD&reCache=1 is one of the examples of me re-doing this CD Shareware collection.
12:43 πŸ”— SketchCow Basically, they got a stack of CDs from this group.
12:43 πŸ”— SketchCow And to be frank, the integration job was shit in a box.
12:44 πŸ”— SketchCow But it appears we got the first 14 million friendster accounts, then.
12:44 πŸ”— SketchCow That's critical.
12:49 πŸ”— Cameron_D We downloaded a total of 20,438,245
12:49 πŸ”— Cameron_D profiles
12:50 πŸ”— SketchCow OK.
12:50 πŸ”— SketchCow Thanks for that info.
12:50 πŸ”— SketchCow While it's sad we lost a lot, we have enough for future generations to see what friendster was about.
12:55 πŸ”— SketchCow http://www.archive.org/details/cdbbsarchive improved the description.
13:00 πŸ”— SketchCow OK, gotta head upstairs, but I'll be back throughout the day to set more processes going, etc.
13:18 πŸ”— jch SketchCow: do you have a kryoflux
13:18 πŸ”— jch i'm seriously considering picking one up in time for the CCC conference in August
13:18 πŸ”— jch I suppose a lot of people there might have something worth storing
13:19 πŸ”— SketchCow Bought one, it'll be waiting for me when I get back.
13:19 πŸ”— jch Ok
13:19 πŸ”— jch Let me know if it is as good as they say
13:20 πŸ”— Ymgve I got a catweasel, fragile beasts they are
13:20 πŸ”— jch I don't understand how it can be completely independent of the disk drive you use
13:20 πŸ”— Ymgve too bad c64 reading needs a drive mod and I'm too lazy to do it
13:20 πŸ”— jch KryoFlux claims they can read c64 without hardware mods
13:20 πŸ”— Ymgve jch: old PC floppy drives are basically pretty dumb
13:21 πŸ”— Ymgve jch: weird
13:21 πŸ”— SketchCow Cat Weasel: Terrible
13:21 πŸ”— SketchCow Kryoflux: Finally great
13:21 πŸ”— jch It's just weird to me that everything goes on in software, w.r.t. reading stuff of disks
13:21 πŸ”— SketchCow FC5025: Excellent, now obsoleted by kryoflux
13:22 πŸ”— SketchCow I'll respond to this window a little later today, talk later folks
13:22 πŸ”— Ymgve jch: i'm pretty sure you still need a mod for reading c64 disks, at least double sided ones
13:23 πŸ”— Ymgve unless you're lucky with the drive you got
13:23 πŸ”— jch I wonder.
13:23 πŸ”— jch (I'm too young to have any real knowledge on 5.25 inch floppies, let alone 8 inch ones)
13:24 πŸ”— Ymgve c64 is special in that it only has one read head, in contrast with a lot of other drives who have to heads
13:25 πŸ”— Ymgve two
13:25 πŸ”— Cowering dude.. the 'mod' is to flip the disk over like everyone else had to do :)
13:26 πŸ”— Ymgve Cowering: no, because a lot of PC drives NEED to detect sector 0 - the little hole near the big one in the center of the disc
13:27 πŸ”— Cowering oh.. true.. stupid PCs
13:27 πŸ”— Ymgve with the disk the wrong way around, the PC drive doesn't sense anything since the IR is blocked, and won't work
13:27 πŸ”— Cowering the dual drive amiga reading hack for PC should work for that though
13:27 πŸ”— Ymgve you also can't use the fact that the PC has two read heads because the PC read heads are offset like 3-5 tracks away from each other
13:28 πŸ”— bsmith093 anyone have the link to the rantmedia torent
13:32 πŸ”— SketchCow http://www.siliconsonic.de/t/flipside.html
13:33 πŸ”— Ymgve SketchCow: yeah, I'm just not much of a hardware mod guy
13:33 πŸ”— Ymgve anyway, for non-original disks, a plain serial 1541 works wonders still
13:34 πŸ”— SketchCow http://forum.kryoflux.com/viewtopic.php?f=3&t=25&p=1589&hilit=c64+flippy#p1589 is the discussion.
13:35 πŸ”— SketchCow Bear in mind, my tollerance for finger wagging geek-superior discussions is about 0.
13:35 πŸ”— SketchCow +/- 0.
13:35 πŸ”— SketchCow I just want to get shit done.
13:35 πŸ”— SketchCow I went downstairs and realized with the time shift I'm an hour early.
13:37 πŸ”— SketchCow So here I am again, making the world better.
13:37 πŸ”— SketchCow Just realized I need to s3 something in, damnit
13:44 πŸ”— SketchCow Wow.
13:44 πŸ”— SketchCow OK, so I never run the --delete option on bitsavers
13:44 πŸ”— SketchCow So I did notice I was seeing double things, i.e. file.pdf in both /apple and /apple/apple_ii
13:45 πŸ”— SketchCow so I thought, oh, they must be removing and moving doubles
13:45 πŸ”— SketchCow So I do an rsync with --delete
13:45 πŸ”— SketchCow Dry run, of course.
13:45 πŸ”— SketchCow They actually deleted a bunch of digitized books, too
13:45 πŸ”— SketchCow Guess someone got to them.
13:46 πŸ”— SketchCow So now I have to write this wasteful script
13:46 πŸ”— SketchCow That will do a find against all deleted items
13:46 πŸ”— SketchCow if nothing with that name is there, put it aside
13:47 πŸ”— SketchCow this historian does not approve
13:49 πŸ”— SketchCow I don't like that I can't trust the thing I'm mirroring to do something acceptable
13:56 πŸ”— Cowering Al does not ever delete things.. i bet all the files are on some other folder now...
14:01 πŸ”— SketchCow I am sure they're on a folder, but not on bitsavers.
14:01 πŸ”— SketchCow I just wrote a program.
14:02 πŸ”— SketchCow 8078378 19072 -r--r--r-- 1 6018 6018 9742343 Sep 22 2004 ./pdf/univOfWiscMilwaukee/UWMCC_1401_OSver2_Jul68.pdf
14:02 πŸ”— SketchCow Original file is in pdf/uwMilwaukee/UWMCC_1401_OSver2_Jul68.pdf
14:02 πŸ”— SketchCow 8078387 19072 -rwxrwxrwx 1 root wheel 9742343 Sep 22 2004 ./pdf/uwMilwaukee/UWMCC_1401_OSver2_Jul68.pdf
14:03 πŸ”— SketchCow See, in that case, the file went from one directory to a another. Fine.
14:03 πŸ”— SketchCow but I'm finding ones where no, in fact it's MIA
14:05 πŸ”— SketchCow Not many, mind you, but enough I have to now run this massive findfest
14:09 πŸ”— SketchCow Example:
14:09 πŸ”— SketchCow HP1000-21XX_CHM_Source_Code_License_final.pdf....
14:09 πŸ”— SketchCow 7067255 216 -rwxrwxrwx 1 root wheel 110350 Jan 22 2008 ./bits/HP/HP1000-21XX_CHM_Source_Code_License_final.pdf
14:09 πŸ”— SketchCow Original file is in bits/HP/HP1000-21XX_CHM_Source_Code_License_final.pdf
14:12 πŸ”— Spirit_ i got a 2G of these now http://www.kimsufi.co.uk/
14:15 πŸ”— Spirit_ i also got a working setup to download files from http://atomicgamer.com/ automated from the shell now, if anyone else is interested (yeah right)
14:15 πŸ”— jch Anybody from the AT going to the CCC Camp in Berlin in three weeks
14:16 πŸ”— Spirit_ i'd love to
14:16 πŸ”— jch It would be ideal for getting shit done
14:17 πŸ”— jch We might even get some new members
14:17 πŸ”— Spirit_ yeah, AT would make a nice lightning talk topic
14:18 πŸ”— jch http://events.ccc.de/camp/2011/Fahrplan/
14:23 πŸ”— DFJustin rsync has a --backup option that could be useful
14:24 πŸ”— DFJustin also you may be able to search through http://bitsavers.org/pdf/IndexByDate.txt rather than traversing the directory structure yourself
14:25 πŸ”— DFJustin I ordered a kryoflux but I do have a lot of flippy disks so I will probably still need to mail them to someone
14:45 πŸ”— ersi jch: I'm going
14:45 πŸ”— ersi jch: You?
16:19 πŸ”— Spirit_ http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/
16:38 πŸ”— Coderjoe i have a kryoflux on the way as well
16:39 πŸ”— Coderjoe and for 3.5", i have a working autoloader i can use.
17:55 πŸ”— ersi Anyone have an idea if IA's liveweb/wayback thing gobbles up other material than just the site? Like pdf's and such?
18:36 πŸ”— ersi Mmmh, bittorrent is "funny".. I got 3 peers connected (one which is slurpin' hard, one who's sipping and one who's sleeping) and I can see 9 in total in the swarm - but they aren't connecting/my client isn't connecting to them
18:36 πŸ”— SketchCow News of the World Archive
18:36 πŸ”— SketchCow 4048.9 / 4175.8 MB Rate: 468.2 / 36.6 KB Uploaded: 9389.2 MB [96%] 0d 0:59 [ R: 2.32]
18:39 πŸ”— ersi Wooho, you're getting there
19:25 πŸ”— ersi SketchCow: http://echoprint.me/data_download <- something to insert into archive.org
19:25 πŸ”— ersi s/insert/ingest
19:39 πŸ”— underscor ersi: Where's the torrent link?
19:40 πŸ”— ersi I'll make one for you if you want
19:40 πŸ”— ersi if you don't, there's afaik only the HTTP download from there
19:40 πŸ”— Spirit_ SketchCow: USB harddisks are so 2008, esata is where its at
19:41 πŸ”— ersi Spirit_: Sure, but there's not that many eS-ATA ports around
19:41 πŸ”— Spirit_ i have two on my 2 year old mainboard
19:41 πŸ”— ersi I was more thinking about to plug in directly, as in chassis
19:42 πŸ”— Spirit_ that would be even easier, my mainboard is 4 internal. i would just need some whatchacallits
19:42 πŸ”— Spirit_ chassis, yes :D
19:42 πŸ”— Spirit_ has
19:42 πŸ”— Spirit_ god
19:42 πŸ”— Spirit_ ENGRISH!
19:42 πŸ”— ersi wat
19:42 πŸ”— ersi Case is proper?
19:43 πŸ”— Spirit_ that would be even easier, my mainboard has 4 internal sata ports. i would just need some chassis accessible from the outside
19:43 πŸ”— Spirit_ if that was what you meant
19:43 πŸ”— ersi no, I meant a port in the chassi/case
19:46 πŸ”— Spirit_ something like http://p.gzhls.at/534865.jpg
19:46 πŸ”— ersi No.
19:47 πŸ”— ersi This -> http://1toppc.com/Merchant2/images/Notebook/Asus/M50/comboport.jpg
19:47 πŸ”— Spirit_ yeah i got 2 of those
19:47 πŸ”— Spirit_ :d
19:48 πŸ”— ersi I don't
19:48 πŸ”— ersi don't think I have one on a single 'puter I got ;p
19:48 πŸ”— Spirit_ damn
19:48 πŸ”— ersi perhaps on one or two of the motherboards though
19:48 πŸ”— Spirit_ well, i chose my mainboard well
20:32 πŸ”— MorbusIff a
21:25 πŸ”— SketchCow Oh boy, let's all jump to esata
21:25 πŸ”— SketchCow because, you know, usb stopped working
21:25 πŸ”— SketchCow poof
21:27 πŸ”— chronomex other day I saw a usb external drive that didn't have a converter in it
21:27 πŸ”— chronomex damn thing had a usb jack right on the disk's ckt board
21:27 πŸ”— chronomex first time I've seen that
21:30 πŸ”— yipdw eSATA harddisks are so 2010, Thunderbolt is where it's at
22:04 πŸ”— swebb1 A single 15k rpm HD can only output 1.5Gbps though (USB 1.0 speed), so what's the point with a single HD and anything > USB 2.0? I could see the need for a RAID or SSD or something, but for a single HD, what's the point?
22:05 πŸ”— swebb1 oops, 1.5Gbps is USB 2.0. My bad.
22:13 πŸ”— bbot_ Yeah, usb 1.0 was what, 11mbit?
22:13 πŸ”— Coderjoe uh
22:13 πŸ”— Coderjoe usb2 is 480Mbps
22:14 πŸ”— chronomex NOT FAST ENOUGH
22:18 πŸ”— tev USB 1.0 was only fast enough to run an external CD drive at about 4x max, IIRC
22:18 πŸ”— chronomex tev?!?
22:18 πŸ”— chronomex how long have you been in here?
22:19 πŸ”— tev I can't remember off the top of my head, probably a week or two
22:20 πŸ”— chronomex crazy times
22:20 πŸ”— tev I had actually been here quite a while back too when I first saw the website, but just briefly
22:20 πŸ”— chronomex aye
22:22 πŸ”— dashcloud did you guys see the new Backblaze storage pods?
22:24 πŸ”— dashcloud http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/
22:32 πŸ”— chronomex interesting
22:33 πŸ”— dashcloud cheaper & better than the 2009 model
22:33 πŸ”— chronomex oh but of course :P
22:34 πŸ”— dashcloud as an addition to that, the company that makes the cases will also sell a fully-assembled model (no hard drives included) for $5500 or so
22:45 πŸ”— dashcloud just under 57k for 1000 TB worth of storage (and another 50k or so to power/etc the beast)
23:02 πŸ”— Coderjoe protocase is selling full systems? I thought they were only selling empty cases
23:03 πŸ”— dashcloud just the cases
23:03 πŸ”— * atg wishes he had someway of justifying all that storage
23:03 πŸ”— dashcloud but apparently some of the parts are tricky to buy in small quantities, or require you to visit some shady places
23:03 πŸ”— atg On the plus side, I didn't realize that SATA port expanders were available as cheaply as the are
23:03 πŸ”— Coderjoe (by full systems I mean with the electronics and backplanes and stuff, just no hard drives)
23:03 πŸ”— atg Dunno how good they are though... :|
23:04 πŸ”— Coderjoe atg: apparently they only work well if your controller uses SiI controller chipsets
23:04 πŸ”— chronomex are SiI chipsets better able to deal with long cables?
23:05 πŸ”— Coderjoe (based on their 2009 installment about the pods)
23:05 πŸ”— Coderjoe chronomex: it had to do with SiI being the originator of the port expansion part of the spec and compatibility issues with other chips
23:06 πŸ”— chronomex ah
23:06 πŸ”— dashcloud http://blog.backblaze.com/Ҁ‹2009/10/12/user-builds-extҀ‹reme-media-server-based-onҀ‹-a-backblaze-storage-pod/
23:06 πŸ”— dashcloud that blog is from someone who actually built the original 2009 model
23:06 πŸ”— dashcloud a user like me or you
23:06 πŸ”— Coderjoe there are comments from me on that blog
23:07 πŸ”— Coderjoe (the blogspot one)
23:07 πŸ”— dashcloud atg: just spin the consolidation angle, and possibly power-savings
23:08 πŸ”— dashcloud you do have a mini-server farm someone in your residence right?
23:08 πŸ”— atg Well yes, but currently putting monnies into networking
23:08 πŸ”— atg friend of mine is doing the storage :p
23:08 πŸ”— atg networking and VMs
23:09 πŸ”— dashcloud if you wait a bit longer, Intel's supposed to be rolling out 10 Gig Ethernet desktop cards next year (single chip, not huge power hogs)
23:09 πŸ”— atg My networking stuff is all gigabit at the moment... I can do aggregrate links which will be fine for just about everything
23:10 πŸ”— atg dual gigabit links agg'd together should be fine for media use / VMs/etc
23:10 πŸ”— atg (yay older dell gigabit switches and ebay)
23:11 πŸ”— Coderjoe link aggregation will not get you a single stream over 1Gbit
23:14 πŸ”— atg Hmm. I hadn't realized that
23:15 πŸ”— Coderjoe the total of all streams can go over, but no single stream will be able to go over
23:16 πŸ”— atg Ah
23:16 πŸ”— Coderjoe and, iirc, it generally uses the mac addresses to keep track of which port a stream is on, and changes it periodically
23:16 πŸ”— atg Still, for almost everything I can envision, an agg'd link from the fileserver to the core switch should be fine for everything else accessing it
23:16 πŸ”— Coderjoe (it's been a couple years since I played with it)
23:16 πŸ”— atg Seeing as everything else will just have a single link, it shouldn't be an issue anyways
irclogger-viewer