[01:16] http://archive.org/details/archiveteam-tvtropes-2012-09 [01:31] oh good, someone grabbed TVT [01:44] A just in case copy. [01:44] 54gb. [01:56] hey SketchCow [01:56] Yo [02:04] i uploaded the last of april 2012 episodes of gbtv [03:18] so is dice on the warrior? [03:23] No, not yet. [03:23] We're still plotting. [03:23] Finding everything geeknet owns [03:54] anyone around know if internet archive's s3 api lets you create folders? [03:54] mine appear to be getting url encoded, e.g. http://archive.org/details/wikitweets [03:54] hrm. [03:54] what is your command? [03:57] i believe i'm doing a PUT to http://wikitweets.s3.us.archive.org/2012/09/19/030901.json [03:57] hmmmm [03:57] I don't pretend to know much about ias3, but that should do what you expect it to [03:58] ok [03:59] 11G JPGMAG-2009-01.zip [03:59] 11gb JPG Magazine, which shut down some time ago. [04:37] Ias3 does not allow folders atm [04:37] cc chronomex / edsu_ [04:38] Ok [05:46] geeknet? what's up with geeknet? [05:48] Sold. [05:48] To Dice.com [05:48] oh crap [05:48] http://www.youtube.com/watch?v=TcxpbhM0DaA [05:57] i know this is part of the plotting that has alread been going on, but is there already an archive of slashdot? [05:57] I fear sourceforge [05:58] huge, and some of the backup procedures have been problematic for me in the past [06:00] i know that slashdot.com does index the stories in a more easy manner to grab [06:01] example: http://slashdot.org/story/1 [06:07] now this is close number: http://slashdot.org/story/174999 [06:08] so we are talking about 175000+ stories just about [06:08] i think it is more like 1.5 million [06:09] oh [06:09] 1457243 [06:09] * Coderjoe tries that [06:09] don't forget, I'm pretty sure that it doesn't give you all the comments for a story on a single request [06:09] hmm [06:10] if there is that many stories then how does 174999 show a story from 2 days ago? [06:11] hmm [06:11] 125000 gets me nothing [06:11] try 125001 [06:11] the story I am looking at for the geeknet sale says story/12/09/18/1457243 [06:12] which is where I got the number, but it didn't work [06:12] 12/09/18/2249200/ [06:13] got that from that from "Feds Add 9 Felony Charges Against Swartz For JSTOR Hack" [06:13] *sigh* [06:13] JSTOR hack... [06:13] "hack" [06:19] someone wanna help this guy http://archive.org/details/proust-panic-download [07:21] i'm on my way to having all of underground gamer forums [07:22] just know some are going to be older then others by a few days [15:41] OK, who wants to take on a downloading of a forum? [15:41] http://mmoquests.com/2012/09/19/vanguard-to-get-new-forums-save-your-old-favorite-posts-vgd-vanguard/ [15:41] http://forums.station.sony.com/vg/posts/list.m?topic_id=59004 [17:58] SketchCow, what forum software [17:59] Not clear. [17:59] Sony never credits anybody [17:59] I've just made a small Lua script for it. [18:00] Now testing to see if it works. [18:02] alard: link plz? [18:02] balrog_: Not yet. [18:52] http://makerbot.com/ [18:52] Quick, go [19:11] Here's the download script for forums.station.sony.com. I think it works. https://gist.github.com/d9559ca9899d7a7f341f [19:11] Is there anyone who wants to run it on the Vanguard forums? balrog_ ? [19:11] alard: ask me later... [19:12] I probably won't be here later, but maybe someone will remember. [19:15] There are more forums of the same type on forums.station.sony.com. Are they going too? [19:16] See the list on the left: http://forums.station.sony.com/vg/user/profile.m?user_id=393 [19:17] They can be saved with the same script. [19:18] Makerbot press event now live. homepage is still broken for me, http://www.livestream.com/makerbotindustries this link here is better [19:20] New location of the Sony script: https://gist.github.com/e046d761b820bfb34de8 [19:20] alard: do we have the recommended wget incantation somewhere? [19:20] I've not used wget with lua hooks yet [19:22] Download-and-build-script of a recent-enough version: https://raw.github.com/ArchiveTeam/cityofheroes-grab/master/get-wget-lua.sh [19:22] The code is here: https://github.com/alard/wget-lua/tree/lua [19:23] kk, got the binary [19:28] alard: Are you keeping that repo up-to-date? I remember at one point it seemed to lag behind the tarballs in the various grabs' downloads. [19:29] alard: do you have a standard/recommended set of args? [19:29] mistym: Yes, it's now up to date, I think. I've restarted the repo, so it's now based on the Wget git repository. [19:30] you hit slashdot. [19:30] alard: Thanks! I'll move my Homebrew build script over to that. [19:30] underscor: Yes, they're at the top of the script. [19:30] http://news.slashdot.org/story/12/09/19/1846211/all-the-tv-news-since-2009-now-available-at-the-internet-archive [19:31] alard: oops, sorry :$ [19:31] heh heh [19:45] alard: Hm, yikes. sed is whining in the ./bootstrap. Have you seen that before? [19:53] mistym: No, I don't know about that. I've merged the most recent Wget commits now, so maybe that helps. (There are no changes for bootstrap.) [20:02] alard: Oh, I see what's causing it. Looks like an upstream gnulib bug actually, so I'll bug the wget maintainers to update their bootstrap script. [20:03] (It's been fixed in gnulib already.) [20:03] Didn't we have that same problem before? Or is this a new bug? [20:03] This is a new one. [20:05] There's a syntax error in the sed line in the warn() function, which was being called because I didn't have a few deps up=to-date in my PATH [20:10] Hey, I wanna post it too! [20:10] http://rss.slashdot.org/~r/Slashdot/slashdot/~3/lJhjRHvOhB4/all-the-tv-news-since-2009-now-available-at-the-internet-archive [20:31] i'm grabbing the utlm.org site [20:37] New feature added to archive.org. [20:37] http://archive.org/derive-wait.php [20:37] Tells you the current wait times for the derivations of items. [20:40] Neat! [20:53] spiffy [21:06] Only hardcores need it, but there it is [21:42] alard: are you aware that you commited an executable binary? https://github.com/ArchiveTeam/cityofheroes-grab -> wget-lua [21:43] SketchCow, pm [21:47] chronomex: Yes, that's the binary for the warrior. (The fastest way to get it there.) [21:47] ah, ok [21:49] hey, what does wget-warc do when you point it at a ftp url? [21:49] ? [21:49] any idea [21:53] It writes the download to the warc file. [21:54] ok, I guess I'll investigate further myself [21:58] http://derive-wait.herokuapp.com/ [23:02] * chronomex hacking together a warc-writing http proxy