[03:18] working on a new viewer for warc files: http://i.imgur.com/O5T5cGL.png [03:20] Very nice. [03:20] neat [03:22] fab [03:26] thanks. I saw one attempt at something similar in Java, but it didn't seem to use a full browser which I think is quite useful. [04:25] http://www.flickr.com/photos/textfiles/10719812953/ [04:31] Thought it was going to be one of the bad ones at first, like these (NSFW-text): http://weknowmemes.com/2013/01/15-hilariously-inappropriate-inspirational-wallpapers/ [05:51] https://archive.org/details/Jeff_Minter_Video_Recording [05:51] The good stuff [07:32] Does alard not come around anymore? [07:35] Alard is busy [07:35] And that's fine, he worked for a couple years on stuff. [07:35] I assume he's getting his other things done. [07:50] The CDX legend appears to be missing 'S': http://archive.org/web/researcher/cdx_legend.php [07:50] which CDX-Writer says corresponds to the compressed record size [08:07] Does anyone know what CDX readers/writers exist? I see CDX-Writer in Python, but it only writes CDX files. I am interested in a CDX reader for Python. [12:06] http://boingboing.net/2013/11/06/archive-orgs-scanning-center.html -> wow [12:10] Wow, what a pic [12:32] Does anyone know if archivebot handles incremental archives, ie: once this one is finished can another be done in the future excluding the files already archived? [12:49] i'm grabbing radioshackcatalogs.com for you guys [12:53] Sum1: At the moment, nope. [15:40] I stripped radioshackcatalogs long ago. [15:47] https://archive.org/details/Castle [15:47] NOT DESTROYED IN FIRE [16:10] found it on google [17:37] Does anyone know why http://urlteam.terrywri.st is 500? [17:46] 503 their servers are either overloaded or misconfigured [18:13] http://www.reddit.com/r/technology/comments/1q3dyk/the_internet_archive_seeks_donations_after_fire/ [18:14] top q is whether it wasn't covered by insurance [18:14] How many scanning centers does IA have? [18:15] maybe SketchCow or someone involved should involve there? [18:19] yeah, someone talking about the work that needs to be done that insurance isn't helping with might help change the pitchfork-wielding few [18:19] man hours, etc [18:20] well, was thinking in general. might be helpful to have someone official answer there. thread is on frontpage of /all [18:20] are the donations for equipment or for the man hours to set up a gigantic scanning center? [18:20] yeah, an AMA would probably be beneficial as well if someone has the time to actually answer the questions [18:21] odie5533_: i'd assume donations just go to the IA's mission in its entirety [18:21] AMAs are crap. just post a blog post answering some key questions. [18:21] could make an irc channel and relay questions into it, if enough are willing to help crafting answers [18:21] pft: that would be strange to ask for a specific amount then. [18:21] AMAs (which are mostly crap) get a lot more visibility on reddit than other things [18:21] i'd use this thread though [18:21] touya: try tweeting it @internetarchive to get their attention [18:21] pft, yeah, true. reddit. *shakes fist* [18:23] FIRE!!!!!!!!!!!!!11111111111 [18:23] DFJustin: done. i rarely use that twatter anymore [18:23] i try to quit twitter. nasty habit. [18:25] ...we came across the word 'twitter', and it was just perfect. The definition was 'a short burst of inconsequential information,' [18:25] Twitter's cofounder sums up the site perfectly: Dorsey has explained the origin of the "Twitter" title: [18:26] haha [18:26] nails it [18:26] uhm /me wanders into -bs [18:26] so, what was burnt in the fire? [18:27] i mean, books? [18:27] n00b256: scanners, a very limited number of materials, and the building [18:27] n00b256: http://blog.archive.org/2013/11/06/scanning-center-fire-please-help-rebuild/ [18:27] not many books apparently. [18:28] kinda ironic, such events like fires are one of the reason we need digital archives, ideally decentralized. [18:30] so, this is historic, first fire in IA buldings? [18:32] perhaps for IA, but it's not the first library fire [18:33] udontsay.jpg :) [18:33] sure [18:33] probably not the first fire in those buildings either, they're in the presidio, right? [18:34] btw what caused it? sorry i haven't read up on it [18:34] a spark in a scanner, possibly [18:34] really? urgh, that's even more ironic then [18:34] iirc they moved out of the presidio years ago [18:35] oh maybe my info is old [18:35] the very tools they were using to prevent the damage of fires caused a fire [18:35] yeah exactly [18:36] somewhat related, i heard that there is serious concern about paper quality since the 20th century. older writings will probably last longer than anything printed in the 20th century. since they started to treat paper with chemicals and various other reasons in manufacturing paper [18:37] Doesn't that just cause the paper to yellow? [18:37] yes, also. but that is a short term thingy (50 years) [18:38] question remeains what happens with 150+ year old paper prints [18:38] this yellow thingy you can prevent by not exposing it to light [18:38] I'd imagine most of them don't actually make it that far and end up in landfills long before that [18:38] unless it's acid-free i think paper only lasts ~100 years [18:38] i have a paper from july 1969 (moon landing) which is still in good shape [18:38] pft: yeah, 120-150 max from what i recently read [18:39] pft: recent paper, or old paper? because there's tons of books that are over 100 years old made with wood paper. [18:39] the "paper" they used pre gutenberg etc. preserves much better, apparently [18:39] i am not into the science of why, though [18:40] http://en.wikipedia.org/wiki/William_Barrow_(chemist)#Significance_to_preservation [18:40] pft: ah thanks, this is what i meant. [18:41] https://en.wikipedia.org/wiki/Slow_fire [18:42] but on the other hand, digital preservation is not better atm, rather worse [18:42] can't expect HDDs to last 100 years at all [18:42] plus you need the right hardware and software to read it [18:43] a good book with a nice paper, last centuries (unmaintained); a book digitized in a server requires continuosly maintenance [18:43] It's not a 100+ year solution, but I imagine around 50 years from now people will have copied the files from the old hard drives. [18:43] there are all kinds of pros and cons [18:43] so I think the most important thing would just be making sure there are multiple copies of the files in multiple geographic locations. [18:44] well, they are working on lasting storage. there is some success with storing data in crytals [18:44] to me one big advantage of storing things digitally is that you have the ability to do validation that the data is still in place and accurate [18:44] which will last like forever if it works right [18:44] pft: you can look at a book and see it still exists. [18:44] real md5sum lol [18:44] true, however it probably takes more manpower and there's more of a risk to the integrity of the book the more often it's handled [18:45] If you lock 50 books in a box, you can be reasonably certain they will stay in the box. [18:45] true, but you can't be certain of their condition [18:45] but the problem with digital data is that you need the right equipment to access it, plus you need to know how it is encoded. (the latter applies to analog data as well, though, need to understand the writing) [18:45] but you don't need any equipment to read a book/stone except your eyes [18:46] touya: that's not really a problem as long as people copy over the previous generation's materials to the new equipment. e.g. copy all the floppies (which is illegal btw don't actually copy floppies). [18:46] encoding in digital is usually so complicated you can't really do it by hand (unless you want to read binary) [18:47] yes true of course. that's what we're doing here.. but it assumes that the infrastructure we have will persist [18:47] also, IA books and in djvu, not transcribed, which is... silly [18:48] this really should be in -bs though, i suppose. but as long as nobody subjects.. [18:48] are* [18:48] we actually have a sort of example case to look at, to see how well transcription from one generation to the next goes: how many of the floppies in existence have been copied to hard drives and shared on the internet? [18:48] 99% of floppies wouldn't work anymore anyway [18:48] or maybe 100% (unless written recently) [18:48] touya: why not? [18:49] i know that *all* of my amiga disks don't work anymore [18:49] data structure destroyed [18:49] are we including what SketchCow has transferred? [18:49] what? I see people uploading recent copies of old floppy games [18:49] after 20 years+ ... most is broken. especially on HD disks (1.44mb) [18:49] 720kb disk, especially 5 1/4 last a bit longer [18:49] important disks as games are copied for sure, your personal data is lost [18:50] none of my personal data is lost [18:50] but i've been careful to take it with me [18:50] odie5533_: yes, but the original disks won't work anymore [18:50] at least none of it i care about [18:50] n00b256: I'm always surprised when I see a new upload of an old game. I figured surely all he games would have been copied, but they haven't [18:50] it was probably copied before [18:50] touya: eh, I'm not sure about that. [18:50] well, try to find some floppy disks that were last used in 1993 or so [18:50] check how many still work [18:50] not many i think [18:51] http://ascii.textfiles.com/archives/3191 [18:51] I have a copy of Tie fighter somewhere, from 1994 [18:51] also http://archiveteam.org/index.php?title=Rescuing_Floppy_Disks [18:51] odie5533_: when did you last try to read it? [18:51] touya: never [18:52] I only have like 2 of 5 xD [18:52] hehe, humor me and try it if you have a spare 10 minutes :) [18:52] well, try to copy them [18:52] doesn't matter [18:52] have to find my old floppy reader... [18:52] oh right. that is a problem now, too lol [18:52] brb [18:52] also, there is a bigger problem, yo can keep real books and the digitized copy, but things like games, youtube videos and so are digital born, so you only can copy them but hardly print them as a book [18:52] i don't even have a vd/dvd drive anymore [18:53] my PC downstairs still has a floppy reader installed [18:53] n00b256: yes, if ever this technology gets lost, all goes along with it [18:53] many written things, too [18:53] most actually [18:54] Kaspersky: .We detect and remediate any malware attack,. even by NSA [18:54] found it [18:54] ogawd. hate hate hate. [18:54] TIE Fighter IBM Disk 5 [18:54] see if you can copy it to hdd? [18:54] Is there a good floppy copy software? [18:54] just use command? copy(cp ? [18:55] and I have the instruction manual [18:55] copy *.* to hdd [18:55] see if it can read it all [18:55] i'd be curious [18:55] alright. bbiab [18:57] at least, if we dont have the alexandria library, and index which what it contained would be useful [18:57] surely we can rebuild a % [18:58] 2 files copied successfuly [18:58] Read and copied fine [18:58] odie5533_: what os [18:58] btw [18:58] http://www.techspot.com/news/50313-hitachi-unveils-quartz-based-storage-data-may-last-100-million-years.html [18:59] Only the best OS ever invented: Windows XP [18:59] odie5533_: okay wow. i'd secretly hoped for I/O erros :p [18:59] try winimage [18:59] also write protect the disk so windows doesn't fuck with it [18:59] my amiga disks are all toast :( [18:59] is write protect where you can see through the gap on both sides? [18:59] most disks can't be read anymore. like they aren't formatted at all [18:59] yes odie5533_ [18:59] can't see thru = can write [18:59] i feel so old :( [19:00] okay. I thought all commercial disks had the chip thing removed, but this disk actually had a movable one. [19:00] it was set to read only though. [19:00] alright. brb, guess I'm making a copy with winimage [19:00] also, I copied some old windows install disks before [19:00] and they worked fine [19:01] most of my floppies are still readable, from dos/windows/amiga 3.5" going back to apple II and c64 5.25" [19:01] i think.. well, most amiga disks were actually 720kb .. and we drilled holes in it [19:01] but some are dead and other people have worse luck, depending on humidity etc [19:01] to make computer believe it was 1.44hd [19:01] works.. but probably won't last very long [19:01] most of my disks were kept in a damp basement with water leaks and they still work. [19:01] it's definitely a "copy asap" situation [19:02] interestingly though, HDDs last even less [19:02] yes [19:02] i mean, keep a HDD for 20 years .. very lucky if it still works [19:02] SSD lifeline is even shorter i think? [19:02] with a hdd the mechanical stuff has to all stay working, not just the magnetic media [19:02] ssd duration depends on writes, i think [19:02] reads are free [19:02] (ish) [19:03] yes, basically a flash drive [19:03] I've heard that ssds aren't designed to last if you just put them in a drawer unused though [19:03] i'd believe that [19:03] http://www.techspot.com/news/50313-hitachi-unveils-quartz-based-storage-data-may-last-100-million-years.html [19:03] this is what we want ;) [19:03] i don't want any of my personal data lasting 100 million years [19:04] heh, not for personal data [19:06] pft why not? [19:06] none of my personal data should outlive me [19:06] it is of no use to people outside my head [19:06] you can't prevent that anyway [19:06] what do you understand for personal data? [19:07] data which pertains to me or that i've created [19:07] family photos? schoool homework? [19:07] that kind of thing, yes [19:07] eh [19:07] I have a boxed copy of Zork Trilogy on the big floppy disk [19:07] why are you going to destroy that? [19:07] but I have nothing that can read them [19:07] it is of no use to anyone except me [19:07] really? [19:08] copyright says 1986 [19:08] odie5533_:i think original images are available on abandonware, though [19:08] dont you like granpa photos? [19:08] hmm my parents have zork disks for the mac, i wonder if those are recoverable. they're probably alerady online somewhere [19:08] i'm not having children [19:08] touya: so it's not worth trying to copy it? [19:08] there is a fair amount of mac abandonware available but most of it is not actual disk images [19:08] same for dos [19:08] came with this sweet zorkmid coin [19:08] odie5533_: didn't say that. [19:09] my parents have a huge amount of mac disks, i should compare what they have with online sites [19:09] see if it's working, if it does, i'd definitely copy it [19:09] so there is value in doing proper rips of original media [19:09] dfjustin: is there a guide anywhere for how to properly rip old mac disks? [19:09] I don't think I have one of the giant floppy disk readers. [19:09] i'm not even sure what image format works on old macs [19:09] aren't there raw copiers? [19:09] well the gold standard would be using a kryoflux or similar [19:10] i am guessing those are not free ;) [19:10] Does anyone here have a Kryoflux reader? [19:10] old mac disks are 800k and can't be read by pcs, later ones are 1.44mb and you can image them with normal pc tools like winimage or dd (just the filesystem isn't readable on pc) [19:10] http://www.myabandonware.com/game/zork-trilogy-2g2#download [19:10] this seems to be sane [19:10] ahh ok, most of what we had were the 1.44 ones, though i don't have a floppy drive at all anymore [19:11] i was thinking id' have to boot up one of the old macs to read them [19:11] what is the stance here btw... for software like Amiga Kickstart ROMs? Afaik they are not free of copyright, it's not legal to share them [19:11] but there is no way of obtaining them legally either [19:12] ugh Â£80 [19:12] pft: for what [19:12] is there a used market in kryoflux where i could buy a used one and resell it once i'm done? [19:12] for the kryflux board [19:12] I assume people that have one would be willing to copy your disks for free for the cost of return shipping [19:12] since they get a copy [19:12] I have a kryoflux but I'm in canada so shipping would be inconvenient for most people, also I don't have time to rip tons of things [19:13] there are some us-based people here that have them [19:13] need a kryoflux for CDs. [19:13] i'd take the time to rip mac floppies if i could upload them to IA [19:13] preferably sooner than later. [19:13] i need to figure out how many need to be ripped though so i'll do that first [19:15] http://archiveteam.org/index.php?title=Rescuing_Floppy_Disks lists some other hardware solutions that are available [19:17] discferret is the only one comparable to kryoflux I think [19:18] yeah, seems that way [19:21] DFJustin: is there anything like kryoflux for CDs? [19:21] since cds aren't magnetic i don't think there's a lot of variance in reading them [19:22] pft: there is huge variance. [19:22] there are sub channels and error correcting data, also I believe some copy protections use the geometry of the disc as well [19:22] ahh [19:22] yeah cds are a huge pain, they're not just block devices [19:22] and there are offsets in how the data is placed on the disc [19:22] afaik there is no hardware solution [19:23] well i know there are issues with things like redbook [19:23] but i didn't realize there was that much variance in data discs [19:23] people are working on software but it's not there yet [19:23] and will probably only work on certain drives in the end [19:24] there are some amazing drive hackers out there. I guess none of them are putting efforts towards this though [19:25] pirating xbox games is more interesting I suppose [19:28] the vast majority of data discs are simple but there are a lot of quirks like copy protection, multi-session, lead-in, cd+g, etc etc [19:41] solution to flippy disk problem listed on that wiki page is to mod the 5.25 drives [19:41] i modded one to add a second index sensor and LED [19:41] and it works fine [19:41] does require some drilling though [19:49] http://www.pixel-anarchy.de/wp-content/uploads/2012/10/diskettenlocher.jpg [19:50] a must-have tool back in the days [19:51] What is it? [19:51] allows the disk drive to read floppy from both sides [19:51] if you stamp a hole on the other end [19:52] there were special stampers to do just that [19:52] it makes the little nick in the plastic on either side? [19:52] yep it basically gives double space [19:52] one side is stamped out by default [19:52] I need one of those for my hard drive [19:52] heh [19:54] allows the disk drive to read floppy from both sides <- incorrect [19:54] that stamper thing allows a double sides (or single sided) floppy disk to be used in a computer which ignores the index hole (the one near the center) so the disk can be flipped to both sides [19:54] it doesn't help with what we need [19:55] what the stamper does is open another write protect hole )the one near the egde) [19:55] so the drive thinks the disk is writable even if it is flipped upside down [19:55] Lord_Nigh: thanks [19:56] some disks have two index holes punched in the sleeve already [19:56] and two wp holes also [19:56] Lord_Nigh: i was too lazy to look it up.. i just tried to make it up from memory [19:56] those disks *CAN* be properly read with kryoflux or discferret or catweasel [19:56] but i remember these stampers [19:56] so are the disks readable on both sides already, and then the puncher makes them writable? [19:57] but those disks with 2 holes in the sleeve are uncommon, maybe 1/30th of the floppy disks i have here have two index holes like that [19:57] odie5533: yes [19:57] Lord_Nigh: i noticed they lose data sooner, too [19:57] we drilled tru 3,5'' disks as well [19:58] tho hm that doesn't make sense [19:58] well the punched wp holes are meant for 5.25" single sided drives so you could read and write both sides of a double sided disk by flipping it over [19:58] as you can't flip a 3,5'' [19:58] touya: drilling 3.5 disks would open up the high density hole [19:58] why did we drill 3,5''? i forgor. [19:58] ahhh! [19:58] so you could use 720k disks as 1.44mb [19:58] right [19:58] Are 3.5" ones already read/writable from both sides? [19:58] but those will be extremely unreliable [19:58] damn, i forgot all of this :( [19:58] yes [19:59] all 3.5 disks are intended to be used as double sided [19:59] there DO EXIST single sided 3.5" drives! [19:59] canon cat used one [19:59] IAI swyft used one [19:59] apple 400k drives are single sided as well [19:59] old macs used one [19:59] 400k [19:59] yeah [19:59] my amiga disks are mostly undrilled though. still they all don't work anymore [19:59] and the brother/roland MOERO drive uses single side as well [19:59] can't read them, even on original hardware [20:00] MOERO is a bizarre format, balrog and I are working on figuring out how the hell to read it [20:00] I think its some unholy nonstandard FM-on-3.5 disk mess [20:00] i.e. single sided single density 3.5 [20:00] which until we found that format i didn't think existed [20:01] its ~180k on a 3.5" disk [20:02] canon cat uses 360k [20:02] IAI swyft i THINK uses 360k [20:02] sorry for off-topic but this pops into my mind.. there was a game called "Starglider II", released for Amiga and Atati ST. The amazing thing was, it was released on same floppy disk [20:02] only the bootloader was different [20:02] the entire game was in machine/hex [20:02] and worked on both 68k machines [20:03] they did something funky there [20:03] cross-platform on a floppy disk, in 1988 or so [20:03] what was even more amazing was a game which was c64 and ibmpc on one 5.25 disk [20:03] amazing shit [20:03] c64 boots from track 18 [20:03] ibm boots from track 0 [20:03] each game has the other game's data marked as used/bad sectors [20:03] heh [20:03] nice [20:04] eh you know kontiki? or what's it called? tcp/ip stack for c64 [20:04] browse all the internet on c64 .. fun shit [20:06] text only? [20:07] i think they could display jpg too [20:07] much like links/lynx [20:07] using some framebuffer alike stuff [20:09] http://www.heise.de/imgs/18/7/2/7/0/4/b895149dab7d2352.gif [20:10] text usually, but you could like click on jpgs and have them displayed [20:10] people still use a C64. [20:11] they don't really [20:11] as a toy [20:12] even breakpoint died :( [20:13] touya: #archiveteam-bs [20:13] That's why we created it, that's what it was made for. use it. [20:14] i knoe i know.. [20:14] /lastlog -bs [20:29] Huh... [20:30] POSSIBLY AT RISK: www.rtvv.es [20:30] Spanish Public Service Broadcaster, closing/closed down [20:35] have set archivebot on it.. (without asking; sorry; blame me later) [20:54] <3 [20:54] It's all your fault! [21:09] :) [21:10] Sorry, I'm a complete noob here. How do you submit a dying site to ArchiveBot? [21:12] just ask here :) [21:12] remember, it's only for relatively small sites [21:13] Ah, thanks. Will do that when I stumble upon something. :-) [21:13] Yeah, I know, [21:13] for larger sites, tell us anyway [21:18] The Dutch public broadcasting organization is doing a revamp of their site, including their broadcast archive/video on demand service which still has free-accessible broadcasts from way back to 2004. [21:19] Somehow I'm afraid they're doing away with that, because of the cuts. [21:21] Our public broadcaster uses Windows Media for their streams and video archives, but without any DRM. [21:22] But the newest additions use Silverlight. [21:23] Main link: http://www.uitzendinggemist.nl [22:39] ------------------------------------------- [22:39] ATTENTION: FOS IS GOING DOWN FOR MAINTENANCE [22:39] ------------------------------------------- [23:37] Is CDX a set format? It seems to only record WARC Response records, but could it be used to index Request records as well? [23:44] or warcinfo records for that matter.