[00:05] It's up on youtube [00:05] or do you want the whole collection of videos? [00:06] It's up on all of the mirrors as well [00:06] I guess he's just interested in if IA has copies, or else perhaps they'll be in the queue in a little while [00:06] geocities.siterip.2009.ArchiveTeam.torrent [00:07] inc.keygen [00:07] right right [00:08] Ymgve, I was wondering because i'm half way done with downloading them from their ftp, and I was wondering if I could upload them to archive.org for them [00:12] ersi, ^ [00:12] If they're uploaded, I think they'd turn up if you search for them [00:13] nope, it's not up [00:14] I bet if the checksums on the files match, they'll be deduplicated.. so if you feel like, up 'em [00:15] Ok, they come with sha1 checksums [00:21] for reasons pertaining to "oh my God, this will take another week" im stopping at 112k stories in another few hours [00:36] script grabs about 1 chapter every 2.5 secs takes another 2secs to convert to epub and save in the folder by authors name [00:37] even if it were one entire story per second, it would still take 55.5556 hrs to grab the first 200k [00:38] its working, im just saying its slow [00:40] one chapter every two and a half seconds? [00:40] that's really slow [00:41] I can pull a chapter off fanfiction.net in around 60ms [00:42] yeah i know, im using a python script, that depends on a whole raft of other scripts [00:42] that's not really an explanation for why it's so slow [00:42] * yipdw tries something [00:43] im also considering the fact that they're not going down, and trying not to hit them too hard, which appears to be a moot point since this script is SO SLOW!? [00:44] when i started, a week ago, i picked an arbitrary number, 200k as the number of stories to grab, im stopping half way through, because this is my single laptop, primary machine, and its been on for a week straight [00:52] bsmith095: well, here's an example [00:52] https://gist.github.com/1573292#file_story_grab.rb [00:52] it can be sped up by (1) accepting multiple IDs, (2) not grabbing the first page twice [00:53] (3) maybe by using more workers, but that gets evil [01:02] im getting error connection_pool when i try and run that [01:02] install the connection_pool gem [01:05] ffgrab.rb:1:in `require': no such file to load -- connection_pool (LoadError) [01:05] from ffgrab.rb:1 [01:05] gem install connection_pool Successfully installed connection_pool-0.1.0 1 gem installed [01:05] same error [01:05] but i did get this WARNING: You don't have /home/ben/.gem/ruby/1.8/bin in your PATH, [01:06] gem executables will not run. [01:06] connection_pool requires Ruby 1.9 [01:06] rvm [01:08] rvm use 1.9.3 [01:09] bundle install, worked, redis-server, runnign, ruby ffgra.rb returs this ffgrab.rb:1:in `require': no such file to load -- connection_pool (LoadError) [01:09] from ffgrab.rb:1 [01:10] already did bundle install [01:10] seemed to grab connection-pool and a bunch of ohter things and instalkked them fine [01:10] there is no bundle for that file, so I'm not sure what you're doing [01:11] nor does that require a Redis instance [01:12] whoops wron project :D [01:13] also, last I checked, nobody has pulled from the read-only redis instance I put up [01:15] i did [01:15] and then left it there, because i had no idea what was going on at that point [01:18] from story_grab.rb:1 [01:18] story_grab.rb:1:in `require': no such file to load -- mechanize (LoadError) [01:18] then gem install mechanize and same error [01:18] yipdw: ruby story_grab.rb 8 [01:20] which ruby [01:20] Ruby 1.9 automatically loads gems, Ruby 1.8 doesn't [01:21] also, I have no idea if that script works on stories that don't have chapters [01:21] my guess is that it doesn't [01:21] actually, it doesn't [01:22] that script also doesn't save anything, so it needs at least that modification anyway [01:22] to e.g. feed the URLs to wget-warc or something [01:22] or shove the text to an ePub filter, whatever [01:22] if you just want to try it out, pick a story that has chapters [01:23] So, I'm learning ruby [01:23] I'll be back later, going home [01:23] Man, what a beautiful language [01:23] ruby1.9.3 and all stories have at least one chapter [01:23] underscor: most parts are, yeah [01:23] I can't believe I didn't play with it earlier [01:23] there are some bits of Ruby that are really zomgwtfbbq [01:23] (in a bad way) [01:23] but no language is perfect etc [01:23] haha [01:23] yeah [01:23] like mechanize, seemed like just what i needed, the docs might as well be in swahili [01:24] its good parts hopefully outshine those warts [01:24] (I mean, I guess they do, otherwise no one would use it ;)) [01:24] Ruby's in a weird spot right now [01:25] at this point, there is a lot of production code written in it, but a lot of people conflate "Ruby" with "the reference implementation" [01:25] and the reference impl has a lot of really annoying characteristics [01:25] rvm? [01:26] some of those characteristics: a GC that loses references to Ruby objects under certain circumstances when you cross from Ruby to C land [01:26] threading code that doesn't make use of multiple processors [01:26] well, they language itself is well-designed [01:26] which I approve of [01:26] yeah, the language itself is solid, or at least the core is [01:28] other implementations like JRuby and Rubinius are doing a pretty good job at addressing the GC and parallelism issues, so [01:28] I haven't run into much in the way of horrible [01:29] I hadn't either until I had to write multithreaded code :) [01:29] heh [01:29] for speeding up a data import [01:29] then it all went to hell [01:29] I got it working fine in JRuby, though, so there was a happy ending [01:29] I had thread issues, a bit, but then I used a connection pool and it's ok now [01:29] but under Ruby 1.9 I'd get inexplicable "deadlock detected" errors [01:29] lightweight shit [01:30] sounds like disasterisk tbh [01:30] I don't know if that's my code or not, but I couldn't find any deadlock conditions [01:30] and it ran fine in a native-threaded environment so [01:30] hm. [01:30] threads are hard. let's go shopping! [01:30] heh [01:31] fucking traffic [01:31] go the fuck home, all you other people [01:31] yeah, I should go hop a train or bus [01:31] well I suppose that's what they're trying to do [01:31] so nevermind. stay at work. [01:32] Chicago traffic patterns influence Seattle's? [01:32] well [01:32] you are clearly within my light come [01:32] *cone [01:32] so we cannot rule it out [01:32] freudian etc [01:32] I guess not [01:32] fuck freud [01:33] einstein++ [01:33] I wish I didnt feel so stupid this week .. a friend gave me an incredibly strong hash cookie last weekend and I'm still feeling it [01:35] * chronomex shrugs [01:36] hash cookie? [01:36] Oh [01:36] I was trying to parse that as a technical construct [01:36] it had a '#' stamped into it [01:36] hah [01:37] hahaha [01:37] actually it didn't but it should have [01:40] anyone wants the files, theyll be in batcave under bsmith as ffnet_dump_and_script.7z [01:40] as soon as i finish compressing them [01:41] no one can access your files on batcave [01:42] but if you tell me when it's finished I can move them to a web accessible directory [01:42] huh well, i didnt know that [01:42] k but im behind a router so how wouold you get them [01:43] I mean after you upload them to batcave [01:43] ah, duh, ok them [01:43] :P man i need more sleep [01:43] lol [01:47] ive added the valid id list as stories and the id list wrapped into urls as link [01:47] both text files both massive [01:48] all in the archive which is still compressing so maybe i should save these notes till after there useful to other people besides me [03:23] You're stealing it wrong: Advanced edition: http://www.atlantis-prophecy.org/recollection/?load=the_list [04:28] damn s3 is pricy... it is almost like buying a new hard drive every month (for the amount of data stored) [06:13] Coderjoe: you also get 3x redundancy across the country [06:15] yeah s3 is very expensive [06:16] and a 99.999999999% data retention rate [06:18] what, not nine fives? [06:18] :'( [06:18] 11 nines :P [06:19] that's hard to believe [06:19] especially given the past year or so [06:19] who had data loss on s3? [06:19] where I think they made 99.99 [06:19] uptime != retention [06:19] i'm speaking only of retention [06:19] false metric [06:20] uptime is fantastic too though [06:20] we pretty much run our entire infrastructure off of it [06:50] http://media.fukung.net/images/17504/d5f6e6850db3607af48ede19154dc83a.jpg [06:51] underscor: can't wait till the day that fits on a microsd card [06:51] :D [07:10] http://www.youtube.com/watch?v=sehBf2266PI [07:13] how did it deal with the uncut page? [07:14] while kinda cool, I don't know that I like the idea of an automated robot being used on delicate materials like that [07:14] http://www.youtube.com/watch?v=cmhIJOqepVU [07:51] ftp://projectd8.org/YouTube/Photonicinduction/ [08:13] that seems like a high estimate for the written works of mankind [08:13] maybe they mean as scans [08:52] k my upload to batcave is done and im going to bed, ffnet_dump_and_script.7z in folder bsmith on batcave if anyone wants it, fanfiction.net stories are in books where they are organized by authors name folders import into calibre to check the tags and metadata which has everything you could ever want [08:53] gnight/ gmorning yall :D [08:54] It appears spambots hit the Google search results of ArchiveTeam [09:14] well, importing proust's sitemap took me from 2900 users to 15,406 [09:14] I wonder how many are duplicates [10:35] Hey again. [10:35] The problem, fundamentally, is that archiveteam's on dreamhost, dreamhost is hacked. [10:35] I clean it out but it comes back. [10:35] It's time to move, I need to find another host. [10:36] A priority for this upcoming week. [10:49] ouch [10:50] I just hate ripping all the wiki out and putting it elsewhere. [10:50] But I shouldn't be. [10:50] It needs its own account. [11:46] http://www.youtube.com/watch?v=VaoxwCayvQM&feature=youtu.be [11:48] Pardon the question, but are you sure that DH is actually the attack vector? [11:49] I get that a lot. [11:49] Yes. [11:49] I am sure. [11:49] I've done experiments. [11:50] Do you have a theory about the exact mode of operation they could have used? [11:51] PHP blows + Dreamhost has massive shared machines [11:52] Apparently DH must have some problems with how their permissions are set up [11:53] Speaking of nothing, I discovered something fun with Wordpress a while back [11:53] If you just upload the WP files to a web location and do nothing, WP has this setup guide [11:54] And it doesn't escape the data that it outputs to its config file [11:55] So you could for example set up a remote MySQL user with a password like, say '); die('compromised [11:55] Probably very few people leave a WP install without setting it up, but it seems like such an oversight [12:00] it's the wordpress-php suck vortex at work [12:01] add drupal for universe collapse [12:01] Or joomla [12:02] regardless, it is a suck vortex. [12:02] It is, at that. [12:03] On that note, I'm still trying to get clearance to roll out that apache patch that prevents spying through symlinks. :/ [12:03] My youtube link!!! [12:03] It's awesome! [12:03] Nobody cares. [12:03] * chronomex watches [12:04] "You need to upgrade your Adobe Flash Player to watch this video" [12:04] oh wait, Chrome works [12:04] weird [12:04] youtube-dl + mplayer [12:05] I don't think I've ever played Dig Dug [12:05] Ooh, I love Dig Dug! [12:05] SketchCow: I don't quite understand what's awesome about that short clip ... [12:07] I see what you were saying about the ambient audio, ahaha! [12:08] But it came out well enough, IMO. Going to shoot more there? That _is_ a really neat spot. [12:08] good visual interest too [12:09] I shot it in a dark room filled with 200 screaming people, and yet the camera can capture both this lit kid under my studio light AND the videogames in the background. [12:10] The content of what he's saying is relatively decontextualized meaningless, if you're focusing on that. [12:10] ah ok [12:10] 200 screaming people, wow. that's a good sound rig you've got then [12:10] Yeah [12:10] The mic is 2 feet from his mouth, to his left, and I have recording at 10%. [12:11] I figured that it was more as a proof of the setup because it's short...I guess I'm getting better at this. :) [12:11] Wow, serious? That's impressive [12:13] It's a little late at night and I have a lot left to do to sit here pulling things out to construct "meaning". [12:14] I'd rather do that when I'm not in the moment. [12:14] But I wanted to do some checking of things out to make sure I wasn't sitting here with, you know, shit [12:14] Wouldn't want to go back, do ANOTHER hour+ of interviews just to find NONE has ANY use. [12:14] This might not have use, either [12:15] Other than, I interviewed a couple who JUST opened an arcade. [12:15] Like, she's 26, he's 27, they JUST OPENED AN ARCADE IN A MALL [12:15] wow, that's actually pretty rare, at least in my understanding of things [12:15] o_o [12:15] And it gets BETTER [12:15] the only arcade I've seen come up around here is the Galloping GHost [12:15] Do you know if they're using vintage equipment? [12:15] Right? I'm the arcade boy, I'm doing this doc [12:15] seattle just got a pinball arcade ~6 months ago [12:15] They opened the arcade, but they CAN"T CALL IT AN ARCADE [12:15] It gets better [12:15] what, landlord restrictions? [12:16] It's because this mall got a department store, and the department store's contract insists the mall not have an arcade [12:16] hahaaha [12:16] hahahaha :| [12:16] So they have it called a "multi-purpose gaming center" [12:16] What the crap. [12:16] which department store is it [12:16] Macy's? [12:16] And stress other aspects... and they can only have 10 arcade games [12:16] super game fun time center [12:16] Anyway, I'm going down to their arcade this month or next, looks like [12:16] what a weird clause for a contract, too [12:16] They want me to come see them, because they're moving to a larger space in mall [12:16] So I will record before and after [12:17] And I met one of the main distributors for Namco, he wants in [12:17] etc. [12:17] I'd love to see it [12:17] Lots of people, good contacts, that's how this all starts [12:17] chronomex: "Pinball" is the magic word. Seatlle just became a very attractive living destination. [12:17] Wyatt: indeed. Seattle Pinball Museum is the proper google term. [12:17] my arcade history actually starts with Bemani, which is a weird starting point, so I actually haven't seen much outside that scene re: arcades [12:18] Wyatt: they got a tiny space (narrow, deep, perfect shape) and put as many pinball machines in there as they could fit along the walls [12:18] $7 gets you in and the machines are on freeplay [12:18] they change em out regularly too [12:19] Crap, crap, I think I'm drooling a little, hang on [12:19] k [12:19] pinball induces slackjaw [12:19] like lockjaw but less socially acceptable [12:22] Good lord is that a beautiful sight. though [12:22] pity they don't have a proper website [12:23] http://www.flickr.com/photos/textfiles/sets/72157628758936913/with/6652371211/ [12:25] What happened with the lighting in that last one? [12:26] I would say you're not keeping their faces within the action-safe rectangle [12:26] Not sure how much that matters nowadays [12:29] Fuck the action-safe rectangle [12:29] I mean, seriously, fuck that thing [12:29] My concern that my film may be lost in the fake-wood border of a 1974 Zenith TV is minimal. [12:30] You might want to care about 4:3 cropping, though [12:30] For those with 4:3 monitors who don't want black borders on top and bottom [12:31] who cares [12:31] those people will just watched all stretched out and shitty anyways. [12:31] people who crop videos deserve the crap that they get out of it [12:31] crap crop [12:32] People with 4:3 monitors, I must remind you, were thrown into the street and shot like dogs during the christmas season of 2004 [12:32] Arcade Documentary (Pan&Scan Edition) [12:32] I mean, I know we want to forget [12:32] But that was us. It was us who did that. [12:32] We have to atone. [12:32] But in the meantime, we'll enjoy our 16:9 proper world [12:33] hey, I have a 4:3 monitor. it's an adjunct to my gigantic 16:9 monitor. [12:33] it's the monitor I use for irc and bittorrent [12:33] I still have a 4:3 monitor here at work and one at home. Also have 16:9 displays and a 16:10 in my laptop [12:33] and sometimes composing emails [12:33] if a video doesn't fit the aspect ratio perfectly I don't get all bent out of shape [12:33] So, of the 10 interviews I shot, only one is majorly out of focus. [12:33] Not bad. [12:33] I have a 4:5 monitor that I use for tall things... [12:33] if a video doesn't fit my monitor, I watch it anyway. [12:34] like when you're viewing wikipedia's list of the world's tallest structures? [12:34] you could just put a 16:9 in portrait mode [12:34] At work I have a monitor that you can flip 90 degrees. I tried it for a while but viewing angles were messed up [12:35] yeah you really need an IPS panel if you're going to rotate it, in my opinion [12:35] nitro2k01: IPS [12:35] Wyatt: where does one get a 4:5 monitor? [12:35] chronomex: Buy a 5:4 monitor and rotate 90 degrees. [12:35] they sell 5:4 monitors? [12:35] hm. [12:36] 1280x1024. [12:36] o [12:37] http://www.flickr.com/photos/textfiles/6652419371/in/set-72157628758936913/lightbox/ [12:37] I am definitely going to need to color grade these, if I use them. [12:37] But this is perfect for me, a complete, 100% nightmare scenario. [12:38] I mean, as truly terrible a situation as it can possily get. [12:38] what did I miss [12:38] Nazis invaded Poland [12:38] oh MAN [12:38] Invention of Steam Engine [12:38] Rock Hudson was gay [12:38] what's your nightmare here [12:38] Ecstacy outlawed [12:39] http://www.flickr.com/photos/textfiles/6652419371/in/set-72157628758936913/lightbox/ [12:39] Documentary footage shot tonight, at the arcade floor of Magfest [12:39] maybe I'm an idiot... what's wrong other than maybe the guy is overexposed [12:39] asians don't look like that [12:40] clearly [12:40] they are much greener than that [12:40] not enough bokeh? [12:40] Oh, the resulting video is actually good [12:40] But this is a room so loud I couldn't hear the questions, I had no real ability to prep for the area, I am stealing location space and want the background to look decent. [12:40] ah. [12:41] I think I could use the video as is, although I likely would do a round of color grading [12:41] Right at the end. [12:41] Because now the footage I'm shooting is so decent, and the quality so high, you actually do have to polish them up. [12:42] http://www.youtube.com/watch?v=VaoxwCayvQM [12:42] You can hear the background there [12:43] But I can't stress how loud it is in there. [12:43] I literally can't hear him. [12:43] Like, I can't hear him at all. [12:43] I'm one of those people who can have his understanding in cross-chatter get lost, and in this case, it's devastating. [12:43] I can't hear what he's answering, and so I am guessing with my next question. [12:43] it must be what it's like for dumb people [12:43] I have a real hard time understanding people in noisy settings [12:44] You hear someone say something, and then you sort of try and look like you heard them and say another thing [12:44] hah, yeah [12:44] not a situation I enjoy [12:44] Even though you literally have no idea [12:44] This is a 35mm lens, by the way. [12:45] Can you get yourself an earbud line from the mic? [12:45] that's a good idea [12:45] Not sure what your audio setup is, so... [12:46] An excellent idea, for this case. [12:46] I like the 35mm in that setting, SketchCow [12:47] Check the closeness of this Mr. Do! [12:47] http://www.flickr.com/photos/textfiles/6646377147/in/photostream [12:47] haha [12:47] it's a shoop, I can see the pixels [12:48] It's a 100mm macro lense [12:48] I have some great macro shots. [12:50] I expect that by the time I'm halfway through this production, I will look at it like I'm watching a 5 year old play with a keyboard [12:50] But for now, I'm very pleased. [12:50] you should interview with that [12:50] just focus on their mouth [12:51] SketchCow: file the mails I just sent you under "tape documentary" before they get lost, btw. [12:52] what did you send!! [12:52] I'm queer for the tape documentary [12:52] Tanks. [12:53] Also, I am shooting with a straight 5D. [12:53] pleasure. [12:53] When I get home, I'm adding the Magic Lantern Firmware. [12:53] That's MUCH better control, TONS better. [12:53] But for now I'm going straight with my 5D, so I'm winging it [12:55] I should bed. [12:55] ditto [12:55] shit is 5am. [12:56] But yeah, magic lantern adds things like MUCH better control on focus, and audio, and the rest of it. [12:56] Right now I'm just using default [12:56] Fine for not, but not fine for, say, Chuck Peddle [12:59] One last unrelated off topic thing [13:00] I had heard of the 14 year old girl deported to columbia, where she does not come from and the language of which she doesn't speak. [13:00] She had run away from home, got arrested, gave fake name related to fugitive (which she didn't know), gets deported through to columbia [13:00] Was there six months [13:00] Now she's returned [13:00] And I'm like "How is she not dead" [13:01] 14 year old girl shoved in country she doesn't come from. Six months. [13:01] I found this: [13:01] The girl was given Colombian citizenship upon arriving in that country, the ICE official said. [13:01] According to the Colombian Ministry of Foreign Affairs, the girl was enrolled in the country.s .Welcome Home. program after she arrived there. She was given shelter, psychological assistance and a job at a call center, a statement from the agency said. [13:01] And it's like, OH. The reason she didn't die is because THE COUNTRY SHE WAS DEPORTED TO ACTUALLY TAKES CARE OF ITS MOST AT-RISK CITIZENS [13:01] That was the small bit [13:01] Here, she'd be a fast-track whoring in a shipping container near a bus stop [13:02] On meth [13:02] Anyway, off to bed, yay documentary filmmaking [13:20] quite the story, that [13:43] I hate when I have a dream, and get a bunch of awesome loot, and then wake up [13:43] The most annoying part is that I'm thinking in the dream "this is not a dream! Cool! [15:21] http://tinyurl.com/89p2f8l [15:25] Good spam. Would kline again. [17:35] SketchCow: I had never seen or heard of vector displays either [17:35] these are amazing [18:26] wow [18:26] damn kids [20:15] so I found a great way to destroy a t1.micro EC2 instance [20:15] do more than one thing at it at a time. [20:15] er, at -> on [20:16] haha [20:16] seriously, I'm uploading splinder stragglers and building Ruby 1.9 on one [20:16]