[00:00] You do know that the x264 encoder only became bluray compliant in 2010 [00:00] I know that [00:01] do you follow doom9 or doom10's forums? [00:01] I still check out the doom9 forums. [00:02] if so, you may have seen the MSU encoder tests: http://compression.ru/video/codec_comparison/index_en.html [00:03] ooh they did a new 2012 report [00:08] aah this is what I am talking about http://x264licensing.com/faq [00:08] Does the x264 license include AVC patent royalties? No, it does not. You will need to get a separate patent license from MPEGLA, see: http://www.mpegla.com for more info . [00:08] and they have no pass through license [00:09] most major companies already have mpeg-la licenses so they can use any encoder/decorder they want [00:12] software patents were never a good idea [00:12] I can agree with that [00:12] but we're stuck with them unfortunately [00:13] When is the patent apocalypse going to happen in the US [00:13] I thought Oracle vs Google would do it [00:14] don't get me wrong I love all the work x264 does but every client I had who had a legal team always bought an mpeg-la license before I started production on their videos [00:14] it was always 'Better safe than sorry' [00:15] If only nut or snow had taken off... [00:15] so according to this question, it would seem you can't have a pass-through license period: [00:15] Does x264 offer a pass-through patent license? [00:15] No. we do not offer a pass-through patent license. It is the sole responsibility of the licensee to get a patent license from MPEGLA, see: http://www.mpegla.com for more info. [00:16] yeah that is what I said a few lines up ^ [00:17] add in the 2008 cisco patent nonse [00:17] and people are afraid of this unknown organization fucking them up in court [00:18] Just another barrier hackers are going to have to go through [00:20] I think you can agree though that it's pretty damn awesome that one of the finest H264 encoders in the world is available as an open-source project (especially with the field being competitive) [00:21] oh yeah I love [00:21] I just hate the bs I have to put up with to use it for professional gigs [00:21] sports events, weddings, etcc... they don't care [00:21] Give it to me on DVD or bluray and that is it [00:21] from their licensing page, it seems dirt-cheap to license [00:22] that is only if you need to embed the software [00:23] I am hoping for vp9 [00:24] or any patent free format that can get the hardware traction to bump off h264 [00:27] too late, h265 is almost here so everything free will be another step behind [00:28] why would people adopt h265? The landscape changed since h264 came out [00:29] that is the question I ask [00:30] Why would people knowingly attach to a codec that is patent encumbered and run by an organization that at one point was going to charge license fees to every h264 users on the internet [00:30] so i have download over 3000k images so far [00:30] only 54 images are the g4 logo [00:30] I just don't understand [00:32] if you don't have patent-free at the very top of your list, h264 is usually the top codec available- it's fast, great quality, supported practically everywhere, and has a great ecosystem [00:32] it's technologically superior and 90% of people don't give a crap about the rest of it [00:33] also know that image urls on the web page is going to be different then what i'm grabing [00:33] it's great that folks are working towards other things, because it keeps folks like the MPEG-LA in check [00:33] page link: http://images.g4tv.com/rimg_606x0/ImageDb3/314633_l/need-for-speed-world-drag-mode-screenshots.jpg [00:33] pirates sure as hell aren't going to care about patents and those are the early adopters of encoding formats [00:34] links i'm getting: http://images.g4tv.com/rimg_606x0/ImageDb3/314633_l/.jpg [00:34] its the same image [00:34] like I said, my whole view about is only in the vein of getting paid for video gigs [00:34] oh crab [00:34] *crap [00:35] the rimg_606x0 are small images [00:35] Open source is always at the top of my list [00:35] without that i get the original image [00:35] omf_: excited for Lightworks ? [00:35] yes. I gave a talk about it right after it was announced. Pissed it is taking so long [00:36] yeah I gave the talk in 2011 and we still do not have it [00:36] Lightworks has great ux [00:38] you didn't jump on the alpha? their twitter page says a few weeks till the public beta release [00:38] I have no video gigs coming up [00:39] what did you think of Kdenlive and Blender's Video Sequence Editor? [00:39] Blender VSE is the best [00:39] feature rich, stable and has a real compositor [00:39] I can crash kdenlive like clockwork [00:40] the blender devs use that product to make money on motion pictures [00:40] that's a shame, since I think they're the 2nd best choice currently- (behind Blender)- would you agree [00:40] that is why its development goes so fast [00:40] I recommend kdenlive to people who want to just edit video [00:41] pititv, cinelerra, and all the other crap [00:41] it is obvious those things are made by people who don't do production video editing [00:41] and that is the real feature of any editor [00:41] OpenShot is interesting [00:41] if anyone has worked with it [00:42] Openshot is junk [00:42] use blender [00:42] junk? definitely not [00:42] blender has proxies and parallel rendering across a network [00:43] joepie91, have you ever used Avid or Lightworks [00:43] once do a few projects with them everything else sucks [00:43] no, but I'm not quite sure what those have to do with how good openshot is or is not [00:43] using avid/lightworks does not magically change openshot [00:44] It shows you all the things you are missing [00:44] that is they [00:44] if openshot isn't suitable for what you're trying to do, fine [00:44] but that does not make it 'junk' [00:44] again, that doesn't make it 'junk' [00:52] I set the bar high for open source software. Maybe I got spoiled because of how good things like linux, gcc/llvm, gnu tools, open source databases [00:52] encoders [00:55] Also I cannot take any video editor seriously that does not have timecode support [00:55] or guaranteed frame precision scrubbing [00:55] imovie has that [00:56] has had that for years and it is super user friendly [00:56] premiere, vegas, etcc. [00:56] these are token features of the beginners versions [00:56] expect that MS Windows video thing [01:00] Has anyone else read the follow ups to Andrew Mason's leaving? His stock used to be worth 1.3 billion and is now worth ~200 million [01:00] ouch [01:22] oh crap [01:23] i think downloading 10000 images maybe be close to 8 to 10gb [01:24] http://imgur.com/gallery/vSgk4 [01:33] so excellent [01:48] SketchCow: i'm trying to get all images from g4tv.com [01:48] i don't think i will get all of them [01:48] but trying to get the originals if i can [01:48] alot of them are 1MB+ [01:50] just know none of these urls will be helpful for wayback machine [01:50] only cause just add $id_l/.jpg [01:51] to the url [01:51] but it could be full url name or just a .jpg [01:51] and i willl get the image [02:04] SketchCow: what happened to me getting permissions to my collections? [03:00] I just got the fail whale on twitter [03:00] Is there a major sporting event I am missing? [03:01] No, but two Justin Beiber tags are trending [03:04] I thought maybe more was going on with spaceX [04:35] looks like the wayback machine maybe of some help [04:36] i'm going to try to find The Daily Nut podcast the g4 had [04:36] the video is still hosted but not playable in browser [04:41] saved: https://archive.org/details/g4tv.com-video14066 [04:50] we may get luckly with some of 14000 ids [04:50] the wayback machine has a lot of them [04:51] saved: https://archive.org/details/g4tv.com-video14131 [04:51] saved: https://archive.org/details/g4tv.com-video14113 [05:04] saved (thanks to wayback machine): https://archive.org/details/g4tv.com-video14162 [05:04] the clip was very broken before [08:13] so i found some of the very old wmv clips of g4 [08:13] the good news is file names are mostly the same [09:28] errr alard did you see my error last night? [09:28] http://pastebin.com/rXvRgixt [09:28] i haven't checked the other users yet this morning, but concidering the tracker hasn't updated I'm presuming they all died in a simular fashion. [09:29] hmmm [09:29] if they did fail, they've disappeared off my warrior screen :< [09:37] re: Lightworks video editor -- takes some wind out of the sails for me: http://jeff.ecchi.ca/blog/2012/11/10/lightworks-is-not-anywhere-close-to-open-source/ [09:50] so guys i screwed up the index for downloading the g4tv.com images [09:50] i got about 1500+ doubles in it now [09:52] Some of that lightworks critique is overblown. Anyway... whatever tool is going to work. [09:53] anyways since the warc is screwed up i'm going to upload the image just in a zip file [09:55] i think its better for me if i just make zip files of this anyways since the warc.gz urls are not going to be used for any of g4tv.com web pages anyways [09:56] this is cause the images are saved as images.g4tv.com/ImageDb3/#_l/.jpg [10:00] i'm going to do a tarball instead [10:01] *tar.gz [10:02] that's a tarball none the less [10:04] its over 5.3gb [10:05] thats just the first 10000 [10:05] its closer to 9500+ [10:05] 9578 [10:09] starting the old wmv uploads i found on way back machine [10:09] https://archive.org/details/g4tv.com-video125 [10:13] ersi, Smiley: I've updated the Punchfork script to have it ignore these connection errors. The latest version should just continue and finish the job. [10:15] alard: Neat [10:15] updating now [10:16] Started downloading~ [10:39] i think my upload speed has double [10:40] i'm holding at 526kbytes a second [10:46] its down to 516 now [10:46] thats amazing [11:32] i got a problem that needs fixing with some items [11:32] i want to change the meta and i don't think i can change it [11:36] can anyone help me? [11:48] ok this link is not working: http://wayback-beta.archive.org/web/20070626122847/http://zdpub.vo.llnwd.net/o2/dltv/episode006/dl.tv.006.mp4 [12:06] alard: when will it update? [12:06] do i need to reboot (i can't remember!) [12:07] - Downloaded: 48340 URLs. :O [12:11] Smiley: You need to either reboot or manually git pull [12:11] .. >_> [12:12] working exactly the same as all previous times [12:12] * Smiley goes for the reboot [12:12] Do we think these running jobs will complete? [12:13] dunno, it's the 'last bunch' - Complexity: Higher [12:14] Ignoring ConnectionError for http://media-cdn1.pinterest.com/avatars/perfectpantry-94_o.jpg [12:14] o_O [12:14] theres an idea - a "git pull" button in the warrior [12:14] That's good @ Re Ignoring ConnectionError [12:15] or a job that makes them update themselves. [12:15] ersi: I've not rebooted... [12:15] or just have the tracker provide the git info, so it can pull a new job and go "oh, I need to update too". [12:16] Guess we could work in an CheckForUpgradedProject Task on most projects that'd block the queue and update and then release the queue for more work [12:16] looks like 10001 to 20000 image set of g4tv will alot smaller [12:16] There's that already Smiley [12:16] If you're on an outdated version, it won't give you jobs - it'll give you "Your project code is outdated. Please git pull or reboot your warrior" [12:16] o [12:16] hmmm [12:16] can't we just have it automatically git pull instead? [12:16] Ignoring ConnectionError for http://media-cdn1.pinterest.com/avatars/perfectpantry-94_o.jpg [12:17] lots of these .... so maybe it did update? [12:17] dude, how can you get on my nerves so much :( [12:17] :D [12:17] you love me really [12:17] I just waiting for these to finish then i'll reboot it :P [12:17] I get that line as well @ Re ignoring ConnectionError [12:17] it's fine [12:17] Starting WgetDownload for Item user-kyoung - Downloaded: 49400 URLs. -- ffs htat might be a while [12:18] Regarding development of warrior/seesaw/projects: yes, we can do pretty much anything. The limiting factor is time and contributions. Like it always has been. [12:19] The warrior checks for updates every hour. [12:19] nod, i'll put a suggestion in [12:19] alard: aaaah [12:19] * Smiley sticks tongue out at ersi [12:19] I thought you changed it :P [12:20] alard: so no reboot required? [12:21] No, not if you're running the warrior. [12:21] Sweet. [12:21] One is uploading now [12:21] kyoung is now over 50k urls [12:21] I went to the gym this morning, and now i'm more hyperactive than ever :/ [12:24] Booyah, climing the top list's bottom [12:47] :O [13:26] Hm, uh oh. ZIP Export failed with encoding errors. [13:34] alard: I filed https://github.com/ArchiveTeam/punchfork-grab/issues/2 [13:56] :< [15:29] "tar: Exiting with failure status due to previous errors" after many hours. and of course, it does not mention those errors again. they just happen to have scrolled way past my buffer. gah [15:38] Â¬_Â¬ forgot to stop my warrior from shutting down Â¬_Â¬ [15:39] O boy, a 2+GB punchforker [15:41] Smiley: user-StaceyCook 2MB [15:48] woot, this user killed the pipeline [15:54] no it didn't, it just blew up that user - and I was stopping the pipeline so that's why it stopped [16:25] D: [16:43] tarting GetItemFromTracker for Item [16:43] Finished PrepareDirectories for Item user-SoniaMichelle [16:43] Received item 'user-SoniaMichelle' from tracker [16:43] Starting GenerateSeedURL for Item user-SoniaMichelle [16:43] Starting PrepareDirectories for Item user-SoniaMichelle [16:43] Finished GenerateSeedURL for Item user-SoniaMichelle [16:43] Starting WgetDownload for Item user-SoniaMichelle [16:43] - Downloaded: 9770 URLs. [16:43] Finished WgetDownload for Item user-SoniaMichelle [16:43] Starting ZIP export for Item user-SoniaMichelle [16:43] Archiving SoniaMichelle to /data/data/projects/punchfork-42fdbb9/data/172090444-1/punchfork.com-user-SoniaMichelle-20130302-153721.zip [16:43] Downloading user profile... done. [16:43] Downloading recipe index: [16:43] - Found 84 cards. [16:44] Building index page: [16:44] - Added 84 cards to index. [16:44] Downloading recipes: [16:44] - Saved 58 recipes.Traceback (most recent call last): [16:44] File "./export-punchfork.py", line 237, in [16:44] pfe.load_recipes(zf) [16:44] File "./export-punchfork.py", line 177, in load_recipes [16:44] zipfile.writestr("%s/recipe/%s.html" % (self.username, recipe_name), soup.prettify(formatter="minimal").encode("utf-8")) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1050, in prettify [16:44] return self.decode(True, formatter=formatter) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/__init__.py", line 334, in decode [16:44] indent_level, eventual_encoding, formatter) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1021, in decode [16:44] indent_contents, eventual_encoding, formatter) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1074, in decode_contents [16:44] formatter)) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1021, in decode [16:44] indent_contents, eventual_encoding, formatter) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1074, in decode_contents [16:44] formatter)) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1021, in decode [16:44] indent_contents, eventual_encoding, formatter) [16:44] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1074, in decode_contents [16:45] formatter)) [16:45] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1021, in decode [16:45] indent_contents, eventual_encoding, formatter) [16:45] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1074, in decode_contents [16:45] formatter)) [16:45] Dude [16:45] File "/usr/local/lib/python2.6/dist-packages/bs4/element.py", line 1021, in decode [16:45] yeah paste fail :/ [16:45] indeedily [16:45] some reason irssi spam limit didn't kick in... possibly br's in it :< [16:45] * GLaDOS kicks Smiley in the shins [16:47] That exact Python Exception is reported btw [16:51] Investigating if I can fix these two issues I've found, we'll see [16:52] k [16:52] the fact the warrior carries on is a blessing [16:52] The warrior don't give no shit about no pipeline error [16:53] True that, but these will need to be fixed to download the 'problem users' [16:54] aye [17:23] Smiley: Forgot to start the warrior? [17:24] Or not done with any users in a while? :o [17:38] nvm, saw you upload one now [18:44] none in a while [18:44] yeah i set it to shutdown earlier for the update [18:44] and forgot it for about an hour [18:49] Hmmm, might have fixed an issue [18:49] trying to see if it'll show up again [18:52] I think a lot of these users have failed - because the ZipExport can take a lot of memory [18:53] At least more than what the Warriors have by default :) Some are 'problem users' or bug-in-pipeline-users [18:54] well i could run some on my 12 gb box... [18:54] "a lot of memory" ~= 50-150MB [18:54] oh [18:55] i think my warrior has like 3gb by default :D [18:55] A warrior typically have 460 something MB. And most people put it to max out on users [18:55] i up'ed it ages ago. [18:55] 6 processes * 100MB = Swappery doo [18:57] i'd set it higher, but if something else needs the memory, vbox is normally onme of the first to get oom'ed [19:09] does the warrior report the state of failed jobs back to teh tracker? [19:09] "I failed this job with error: xxx" [19:14] I havn't checked, but AFAIK no - it doesn't. But there's a limit in the tracker on how many items you may claim. I'm not sure if al ard releases them manually or if they're autoreleased though [19:15] fuck, okay - I still havn't fixed it :P [19:19] As a non-programmer I don't know how much use that'd be. [19:20] pretty sure alard or another admin (chronomex?) has to release...posterous might auto release havent read up in that conversation yet but i can release there [19:20] S[h]O[r]T: he assigned them to me and ersi [19:21] we are pushing them through atm, and we're reporting errors as they occur. [19:21] Don't worry about it [19:21] some really small ones got missed. Bit random really [19:21] just throwing facts around :p [19:21] Most are a few hundred meg, but I've had a few 10-30Mb ones. [19:22] Yeah, don't worry man [19:22] Something is always better than nothing [19:22] o_O Who's worried?! :P [19:22] I'm just speculating on randomness and seemingly driving you insane. [19:22] You seem worried since you're asking about so much things [19:23] * Smiley looks for question marks :D [19:23] s/much/many/ [19:23] "What about this, what about that" [19:23] I'm not worried, just hyper.... lots of crap going on in life atm [19:23] this is helping keep my mind off things that people really shouldn't have to think about. [19:24] those things I don't talk about are the ones I'm actually worrying about :/ [19:25] make sense now? [19:26] so you're saying I should ignore you? Ã²_Ã² [19:26] hmmm, this requires pm [19:26] can I Pm you? [19:26] sure [19:27] I'm just stating stuff I've seen, and coming up wth suggestions for improvements [19:28] Just don't feel I require any replies, but I do like replies and discussion :) [19:29] Questions are about growing, learning, understanding, ideas, knowledge. [19:30] argh, fuck encodings and python2.X [19:34] uploaded: https://archive.org/details/images.g4tv.com-1-to-10000_l-images [19:42] tef, soultcer: Anyone of you feel like taking a look at https://github.com/ArchiveTeam/punchfork-grab/blob/master/export-punchfork.py#L177 ? Me+encodings=fuckup. You'll get a UnicodeDecodeError exception at this line occationally [19:43] do you have a traceback [19:43] yeah [19:43] https://github.com/ArchiveTeam/punchfork-grab/issues/2 [19:44] what version of python are you using ? [19:44] 2.7.1 [19:45] ok that should be modern enough [19:45] you can put a print repr(self.username), repr(recipe_name), ... repr(....) above it [19:45] sure thing [19:45] then if you see one the strings that start with u"..." [19:45] put a .encode('utf-8') after it [19:46] thanks tef, I'll get right on it [19:46] UnicodeDecodeError [19:47] printing %r would be repr() right? [19:48] I think, I just go repr() around strings most of the time, it's debugging code [20:00] uploaded: https://archive.org/details/images.g4tv.com-10001-to-20000_l-images [21:21] if tar's only error is: [21:22] tar: Exiting with failure status due to previous errors [21:22] tar: oystatic.ignimgs.com: Cannot stat: No such file or directory [21:22] ie, i told it to add a directory that did not exist. does that mean the tar it wrote is correct? can i test somehow (in a faster way than just reTARring without telling it to add that dir)? [21:43] tar -tvf and compare against a file list [22:00] hmmm [22:00] I'm going to ask a question. Don't explode plz ersi [22:00] np [22:01] How many posterous accounts between us were left? The tracker says there is 40 out.... yet I presume the warrior only claims 6 at a time... [22:01] If you have the rest, I'm gonna switch. [22:01] s/posterous/punchfork [22:01] I assume the ones that are assigned to us, aren't showing in that number [22:02] yeah, I thought that maybe the case. [22:02] * Smiley leaves it to see if it gets more jobs [22:02] 46% so we'll know soon enough. [22:02] I know there's jobs left, since I've failed a lot of users because of the UnicodeDecodeError [22:03] same [22:03] well, not that exact error but others. [22:03] I think I might have fixed/worked around it, waiting for a confirmation though [22:04] got a few damn large users coming down 50k+ URLs [22:04] yup, tho one of those uploaded for me earlier [22:05] Hense why I'm kind of babysitting the warrior now. [22:46] 73k URLs >_> [22:47] :o ok u win [22:49] punchfork.com-user-swedishfish-20130302-212722.warc.gz [22:49] funny considering I'm a swede [22:54] :) [22:54] * BlueMax hits Smiley with a swedishfish [22:54] 2x32k+1x42k atm and some small ones [23:39] Gears of War 2 Gameplay Demo: https://archive.org/details/g4tv.com-video26929 [23:40] Fable 2 Gameplay Demo: https://archive.org/details/g4tv.com-video26928 [23:41] Resident Evil 5 Gameplay Demo: https://archive.org/details/g4tv.com-video26927 [23:42] Cobra For President: https://archive.org/details/g4tv.com-video26880 [23:49] haahaha, yet again SketchCow demonstrates that he needs more middle fingers: http://www.flickr.com/photos/kennethreitz/8518208951/ [23:50] chronomex: Could you pause the posterous tracker? [23:50] I don't know how. [23:50] I just pay for the box [23:51] 4 punchforkers >60k urls >_> [23:52] :\ [23:52] chronomex: Do you know nginx config well enough to make it return 404 on any request to /posterous? That should do the trick [23:53] Actually, any POST request to POST /posterous/request needs to receive a 404 [23:53] hrm [23:53] is it using nginx? [23:54] * chronomex investigates [23:54] well, I'm rearranging my apartment naked today [23:54] Sounds dangerous [23:55] I'm careful. [23:55] it was a competition between the laundry and the furniture [23:55] furniture won [23:55] twe [23:57] so soultcer [23:57] 404 on all requests to /posterous/request ? [23:58] yes [23:58] Microsoft E3 '08 Press Conference Archive: https://archive.org/details/g4tv.com-video26931