[01:10] How big is rawporter? [01:10] SketchCow, looks like it'll be about 12GB [01:10] Really? [01:11] * SketchCow looks around [01:11] * SketchCow gets a box of macaroni and cheese [01:11] * SketchCow dumps mac and cheese packet on floor [01:11] Here, store it in this [01:11] lol [01:16] SketchCow: sure you know this, but somehow Angelfire and Tripod are both still around and maybe even thriving- yet Geocities wasn't able to make it when it was the biggest of the three of them [01:20] Kinda like this SketchCow? https://imgur.com/StKMhqD [01:21] 12 Giga-Bites [01:22] heh [01:23] vantec: Exactly that [01:27] am I the only person who just found out about UAS (USB-attached SCSI)? http://hansdegoede.livejournal.com/14660.html You can finally get the full performance of a disk/SSD when hooked up over USB3 (provided you have an enclosure that supports UAS) [01:28] dashcloud, only really useful for SSDs, you're never going to have USB3 bottlenecking a hard drive. ;) [02:47] http://discimage.tumblr.com/ [02:47] Curated disc images [02:54] shiny [02:54] Literal. :p [02:54] also, slightly dizzying [03:00] ooo, Cultures! [05:46] https://www.youtube.com/watch?v=sKIOqJns5N8 [05:46] Someone youtube dl that before it dies [05:48] got it [05:50] Thank youuu [06:08] boo they closed the S3 service it seems [06:09] i have 23874 of 39685 of the items [06:09] what?! [06:10] i still see it up [06:10] e.g. http://rawporter.s3.amazonaws.com/uploads/it86ue4m83dphc.flv [06:10] yeah, i got a forbidden [06:10] lemme check why [06:11] i'm at 34604/39685 [06:15] did you pull in random order? [06:15] it crashes in the AWOL folder [06:16] might just skip that one [06:16] or, since there are two of you, did one of you reverse your traversal? [06:17] mine has some hate now, error 418, 416 [06:17] first have to drive to work again [06:46] If an archive team member in England wants to go to this, I can help pump up your proposal. http://failureinthearchives.wordpress.com/ [07:45] someone please mirror the torrents from http://chriswhong.com/open-data/foil_nyc_taxi/ to archive.org. highlight me _after_ you did. thanks! [07:46] i mean to contents of the torrent, not the .torrent files of course ;D [07:48] schbirid: what's the difference? archive.org downloads the torrent content if you upload the torrent, is that not good enough? [07:48] Nemo_bis: i had no idea, that's crazy [07:48] * Nemo_bis now wonders if the highlight request was respected [07:49] heh [07:49] let me try that [07:49] ok [07:54] let's see what happens https://archive.org/details/nycTaxiTripData2013 [07:58] mm... that looks interesting [08:10] I'm uploading to IA torrent client :D btw schbirid did you add both of the torrent files? [08:25] someone who isn't going to sleep could grab a copy of http://delimiter.com.au/2014/06/18/delimiter-coming-natural-end/ [08:29] what, just natural? not an organic, free range, non-gmo ending?! [08:33] apparently [08:34] Cameron_D just put delimiter into archivebot [08:35] good [08:36] Yeah, looks like the site will stick around but won't be updated, but still worth grabbing [09:02] deathy: yeah, both in one to see what happens [09:11] deathy: I'm not sure two torrents work, IIRC it was necessary to give the torrent the same name as the item [09:12] ah no, it seems it's done with the first and 20 % with the second :) https://catalogd.archive.org/log/316848601 [09:12] sweet :)) [09:13] what a leecher! 55m18s | .. Percent Done: 93.3% Peers: ^ 1.37 MB/s to 6, v 4.08 MB/s from 13, of 14 (Ratio: 0.34) [12:43] Nemo_bis: the files were downloaded but they are not listed https://archive.org/details/nycTaxiTripData2013 :\ [12:46] schbirid: that's normal because you chose mediatype text, they're in https://ia802501.us.archive.org/1/items/nycTaxiTripData2013/ [13:12] Nemo_bis: it did that all by itself. i used the browser uploader and even let the collection at "media" by default [13:56] so i maybe able to get video from here: http://www.click2houston.com/sitemap/video-20110701.xml [13:56] i couldn't use youtube-dl [13:57] but i grab the video link thru httpfox and here is the link to the first video: http://ib141804.ib-prod.com/p/557781/sp/55778100/serveFlavor/entryId/0_8u87aii9/v/1/flavorId/0_gspvcjay/name/a.flv [13:59] based on want i can tell http://ib141804.ib-prod.com/p/557781/sp/55778100/serveFlavor/entryId/ maybe in every url [14:00] you then take the part at the end of the video:player_loc url: 0_8u87aii9 [14:01] *video : player_loc [14:04] looks like the stuff between flavorid and /name/ is not in the xml [14:28] http://www.marketwired.com/press-release/blippar-acquires-layar-creating-worlds-largest-ar-userbase-1921802.htm [14:29] Blippar buys Layar [16:48] http://www.securitycurrent.com/en/writers/richard-stiennon/cloudflare-acquires-cryptoseal [16:50] DDoS ALL THE VPNS! [16:52] woop woop woop off-topic siren [16:56] exmic: your siren is sensitive today :P [17:11] It's true, though [17:35] mmm, delicious roast beef on sourdough [17:38] SketchCow: i'm starting to upload Bobby Blackwolf Show: https://archive.org/search.php?query=creator%3A%22Bobby%20Blackwolf%20Show%22&sort=-publicdate [17:38] i need to use dos2unix just to get the xml data to upload [19:58] rawporter is shaping up to be 30GB+ [21:14] i'm gonna have to stop my rawporter grab, my estimate is that it's going to be >50GB, which i can't do right now [21:15] so the ones i haven;t grabbed are tail -n+35900 urlList.txt [21:15] *haven't [21:31] mine is still running, have some 600GB free on that box [21:31] great! [22:02] okay [22:02] panic [22:02] http://freecode.com/about [22:02] freecode.com? [22:02] looks like it's going to require urgent saving [22:02] this is pretty much a notice of death [22:02] "we put the site on static mode" [22:02] "because not much happening" [22:03] "The site contents have been retained in this static state as a continued path to access the linked software, much of which is on self-hosted servers and would be difficult to find otherwise." [22:03] cc SketchCow yipdw exmic [22:04] hmm [22:16] joepie91_: oh yeah [22:16] I wonder if we can just archivebot it [22:16] well, probably not [22:16] luckily it has a URL structure that isn't horyshitinsane [22:17] Just recurive wget it. :D [22:17] probably just split it up by project [22:19] actually [22:19] http://web.archive.org/web/*/http://freecode.com [22:19] maybe no action required [22:19] yeah, unless someone can show a deficiency in the Wayback grabs, I say let it be [22:20] clicking around, this seems pretty complete [22:20] oh, some of the download URLs have bad robots.txt rules [22:20] ok [22:20] so maybe just grab all the download links for starters [22:21] yipdw, wouldn't those be 90% of the total size anyway? [22:21] I don't know, I didn't run a size check [22:21] there should be a full run anyway [22:21] for the stuff that is missed but unnoticed [22:21] (and hey, it's static anyway, heh) [22:22] SN4T14: that said, freecode didn't appear to host the downloadable archives, just the project metadata [22:23] yipdw, then someone here will probably just get a complete archive of it, text and metadata isn't that big. :p [22:24] sure, that's fine [22:24] I'm just not panicking over it, since the Wayback grabs of it are pretty extensive already [23:52] you guys should read Constellation Games, if you haven't already