[02:22] Hoorah [02:22] 1. Found 20,000 more artist mp3s to add to IUMA [02:22] 2. Someone stole one of my credit card IDs! Caught! Bitch! [02:23] SketchCow, nice [02:28] oh christ [02:28] I spent hours today chasing down a problem in Vagrant's provisioning of VMs [02:29] turns out it's because Vagrant depends on Ruby 1.9 hash ordering and I'm running it using Rubinius in 1.8 mode [02:29] a four-character fix gets things working [02:29] TL;DR: fuck software [02:44] We hope that you are enjoying your use of AWS. We wanted to let you know that your Amazon S3, Amazon EC2 and Amazon EBS introductory 12 month free usage tier for account number [redacted] is set to expire on May 31, 2012. Your resources will continue to run once your free tier has ended, but will be charged at the standard, pay-as-you-go service rates as shown on http://aws.amazon.com/pricing/. [02:45] :( [03:07] godane: whee. your item is still deriving after 4 hours [03:57] it must be have problems with deriving it [03:57] there both 1 hour long videos [03:57] i mean all [04:06] three hours. ok, that's probably going to gake 5 to 6 hours [04:17] >> LORD_BYGON [04:17] Description ends at line 10... [04:17] Description starts at line 489... [04:17] OOOOOOO lord_bygon IS NEWWWWWWW [04:17] ZZZZZZZZZZZZ while it gets its shit together..... (Grabbing LOOT.B) (2551) [04:17] We make it a pretty princess. lord_bygon [04:17] Princess looks so ugly when she cries. No mp3s for lord_bygon [04:17] OOOOOOO lord_cyrus IS NEWWWWWWW [04:17] >> LORD_CYRUS [04:20] a part of me always wonders if you guys could have archive megaupload before it went down [04:21] only cause megaupload had like 25PB vs archive.org being less then 6PB from what i have read [04:21] godane: It's 25 *peta*bytes [04:21] i know [04:21] And even if, by some miracle, we got it, it'd be impossible to curate [04:22] Since AFAIK there was no useful metadata - just filenames and maybe upload date [04:22] So it'd just be this 25PB pile of data [04:22] No [04:22] archive.org put a bid in for the equipment [04:23] it was not just megaupload anyway [04:23] alot of other fileservers have files removed [04:25] I'm actually very curious how the feds plan on dealing with it to prove infringing content [04:26] It'll be a neat IR trick [04:47] SketchCow: even if the feds do wipe it having 25pb of storage space can't be bad [04:49] unless i missed something the government doesnt want the data [04:49] pretty sure they havent paid a fucking dime for anything [04:50] the feds actually wanted carpathia to wipe it [04:50] yeah but the judge said not uh..except that was about a month ago [04:51] right [04:52] but the govt wanted the evidence wiped, along with any software they may have. probably in hopes that no matter the court outcome, it would be difficult to get back up and running [04:52] I was wondering about that - how much of the evidence falls into public access? [04:54] looks like someone wrote a recent article and escapist republished too but i guess nothing new in terms of a decision about it. only something today about kim dot com not giving over encryption passwords unless they strike a deal with him for some other data [04:54] http://webcache.googleusercontent.com/search?q=cache:http://aceds.org/news/25-million-gigabytes-brings-prosecution-megaupload-and-its-eccentric-founder-standstill [04:54] i cant imagine carpathia still has them online [04:54] they must be unplugged in storage or something [04:55] most likely still on without transit [04:55] or at least without dns [04:56] large systems generaly don't like being power cycled in their entirity. [04:57] i dont doubt it cost them 9k a day [04:57] not going to take that loss [04:57] that's not just power and cooling, but lost opportunity costs on the hardware and rack space [04:58] yeah [04:58] plus i bet they have bandwidth commits they arent meeting based on what megaupload used [04:59] depends [04:59] if MU was just getting a crossconnect from a network provider, rather than getting transit from carpathia, carpathia wouldn't really be paying that bill [05:01] i imagine they had to have at least a 10Gbps with more than 1Gbps commit. be it megaupload directly or through carpathia someone isnt getting paid for that :p [05:03] i know they lease the hw from carpathia but i imagine they had to pay some money upfront for all of it. i wonder if its all paid off [05:05] oh, I'm sure the actual purchase costs of the hardware are paid off. [05:05] but as MU was renting the servers, they still belong to CH, and CH can say they're losing money by not being able to wipe and rerent the servers [05:05] yeah [05:06] Load average: 114.48 40.36 15.11 [05:06] hahahahaha [05:06] I love saving tabblo [05:06] * S[h]O[r]T reads up for helping [05:07] what kind of file sizes are we looking at [05:07] 1-6MB, 1000 per section at most [05:08] worst case, like 6GB [05:08] but a lot less than hthat usually [05:08] that* [05:08] (per claimed number) [05:08] why claim such a small range then? [05:08] very slow? [05:09] no [05:09] feel free to grab more [05:09] or am i not understanding the ranges correctly [05:09] they're in sets of 1000 [05:10] 100-499 is really 100,000-499,999 [05:10] etc [05:10] ah [05:11] ill claim 550-599 for now? [05:11] sure [05:11] k [05:11] Probably a nice change of pace from FP [05:11] then will go from there once i see how this goes [05:11] then you want something like [05:11] well apparently my other box is still doing mobileme but dont feel like im conributing lol [05:11] Where a range of 500 means 70GB [05:11] ./dld-tabblo-zip.sh $q $i & [05:11] done;done [05:11] for q in `seq 550 599`; do for i in 0 1 2 3 4 5 6 7 8 9 ; do [05:11] but hey im still in top 10 [05:12] is there more fileplanet to do? [05:12] yeah lots [05:12] underscor: Lots more [05:12] oh okay [05:12] I should probably help then [05:12] is there a tracker or what? [05:12] Not really; we're doing it manually [05:12] Are there instructions somewhere? [05:12] But it takes so long to make progress it's fine [05:12] http://archiveteam.org/index.php?title=Fileplanet [05:13] if i register on the wiki can i make changes or are these articles protected? [05:13] S[h]O[r]T: You can make changes [05:13] eg tabblo or fp [05:13] meant to ask the other day. might as well do that + announce so dont have to bug/rely on u guys hehe [05:14] Can you register? [05:14] atal error: Call to a member function userCannot() on a non-object in /home/archivet/public_html/extensions/TitleBlacklist/TitleBlacklist.hooks.php on line 86 [05:14] nope lol [05:14] That's why we've been sharing my account, lawl [05:14] haha [05:15] http://www.mediawiki.org/wiki/Extension_talk:TitleBlacklist [05:15] oh shit [05:15] S[h]O[r]T is about to pass me [05:15] time to fire up all thrusters [05:15] (mobileme) [05:15] :D [05:16] i still want to know wtf is with the old listings i have in tamer [05:16] Pity FP doesn't add to that - S[h]O[r]T's got a couple hundred GB there [05:16] i guess once its done someone will go through them [05:16] its all noted on the wiki so :P [05:16] Client Version Owner Status Updated Said hello First seen Target [05:16] c-1422 5 short 100 percent full 90934m ago 0m ago 91203m ago [05:16] wow [05:16] that's impressive [05:17] all the ones at the top idk what it is [05:17] hahaha [05:17] you just have some box somewhere [05:17] and no idea [05:17] i thought it might have been from a box i stoped using but i swear i closed them out [05:17] lol [05:17] plus yeah [05:17] they are saying hello [05:17] well, they're still running hello [05:17] so they're on [05:17] maybe i should ps faux on this box [05:17] cause its nothing in a screen [05:17] the 4 i have running are the ones below [05:18] i mean theres that 894 % full one lol [05:19] ahh i think i see them [05:19] i guess i can just kill -9 em? [05:20] its a bunch of seesaw-s3.sh [05:20] why not leave them? [05:20] :P [05:20] i dont think they are doing anything? [05:20] shouldnt it be like emptying the bucket or something [05:22] guess i can jus leave them and once everything is done follow any instructions im given :p [05:22] Is fileplanet on akamai? [05:23] Akamai? [05:23] I'm puling 166Mbps from akamai, trying to figure out if I have a rogue mobileme or if it's fileplanet [05:23] It's a cdn [05:24] jeez, ign has a /20 [05:24] anyway [05:24] no idea guess i could check where my FP traffic is going [05:24] man, this seems slow [05:24] (fp) [05:25] akamai is level3? [05:25] fp isn't on akamai [05:25] just checked [05:25] the IPs of the fileservers whois to IGN, not to akamai [05:27] can u update the wiki for the range i took when u get a sec? [05:27] What range is this? [05:28] hah, downloading fileplanet at 137Mbps [05:29] feels delicious [05:29] Christ [05:29] Are you downloading directly to archive.org? [05:29] mmm well i guess i should have realized that for script would run everything in the background [05:30] nope, this box is on a different network [05:30] 10gigE though [05:30] omg holy shit ig goes [05:30] *it goes [05:30] and akamai has servers like 8 feet away from it [05:30] (in chicago) [05:30] 550-599 on tabblo http://archiveteam.org/index.php?title=Tabblo [05:31] 2012-05-22 05:31:29 (29.9 MB/s) - `www.fileplanet.com/222027/download/Metro_Last_Light_Uncut_E3_Demo.mp4' saved [287682319/287682319] [05:31] hahaha [05:31] well i bet those files are on nicer servers being as large [05:32] I just hope I don't fill up the 2TB drive [05:32] underscor the screen of running that script is crazy lol [05:32] cant really track the status [05:32] isn't it great :D [05:32] jus a shit load of curls [05:32] it's fun to watch [05:32] yeah [05:32] what's your system load? [05:32] :D [05:33] 00:42:47 up 740 days, 10:35, 4 users, load average: 1.04, 0.75, 0.60 [05:33] oh, load hasn't jumped yet [05:33] model name : Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz [05:33] nice uptime though! [05:33] yeah [05:33] shutdown -r now [05:33] ;D [05:33] im going to have to take it down sometime since i want to rebuild it [05:33] but just no spare time [05:33] Windows box I'm on [05:33] my dmesg is full of Not cloning cgroup for unused subsystem ns [05:34] Uptime: 13:05:16:39 [05:34] woot [05:34] Core2Quad Q6600 [05:34] translate that to english for me [05:34] I have a 6700 on my desk, just too lazy to put it in yet [05:34] 13 days, 5 hrs, etc [05:34] ah LOL [05:34] 13 days is a lot for windows! [05:34] I once had a XP box up for 100 days [05:34] blashpemy [05:35] It fell when the CDR shit itself :( [05:35] you must reboot and pray to the gates shrine every 3 days [05:35] or else [05:36] mmm, fileplanet [05:36] 10 gigs in 9 minutes [05:36] delicious [05:37] er [05:37] 4 gigs in 9 minutes [05:37] what range did u do [05:37] looked at the wrong number [05:37] lol [05:37] 222000-222999 [05:37] lol dang [05:38] 4.2 now [05:38] this is mesmerizing [05:40] What ID are you up to now? [05:41] uh, idk [05:41] let me look [05:41] http://imgur.com/MdQrv [05:41] :D [05:41] Can you tell where I started downloading mobileme on that box, then stopped, then started again, then stopped? [05:41] Hah [05:42] shaqfu: 222070 [05:42] and [05:42] 5.7G . [05:42] damn [05:45] seems like from what i can read getting some errors with the tabblo stuff? [05:46] some operation too slow and cannot find or open data/tmp. or something [05:46] scrolls too fast [05:46] maybe i should just kill them and run 1 for now? lol [05:47] nah [05:47] I think that's just tabblo messing up [05:47] known thing [05:47] it retries until the cache server doesn't time out [05:49] JUST SIT THERE AND TAKE IT [05:50] speaking of, I just watched Human Centipede 2 with a friend [05:50] so fucked up [05:50] eww [05:51] although, I suppose it's better with friends [05:51] yea [05:51] S[h]O[r]T: what range did you take again? [05:51] 550-599? [05:52] added here: http://www.archiveteam.org/index.php?title=Tabblo [05:52] if that's wrong, let me know [05:52] going to snooze [05:52] yup thanks [05:56] http://i.imgur.com/uIJNy.png [05:56] Damn I use a lot of internet [05:56] This is why I love FiOS :D [05:56] (that's the past week) [06:01] goddamnit underscor [06:01] I get 200GB a month and you use 2TB both ways?! [06:01] I get 250GB. 10 bucks per 50 GB over. And a bs modem that has no bridge mode. [06:03] :( [06:03] BlueMax: Should move to not-an-island [06:03] :D [06:04] fuuuuuuu [06:04] <3 [06:05] wheee [06:05] maxing out 2 100mbit lines on mobile me [06:05] and 170mbps on fileplanet [06:06] and 50mbps on tabblo [06:06] such a good feeling [06:07] best domain name [06:07] .573a6d1e5d75df14c27fda2c49f62662.fr [06:07] er [06:07] 573a6d1e5d75df14c27fda2c49f62662.fr [06:40] underscor load average: 287.53, 191.51, 101.91 [07:18] (trying to fetch everything down to an instance so I can rsync it home, since I can't seem to get s3cmd's sync command to work properly) [07:18] grr [07:18] i get the feeling that (amazon) s3 is artificially limiting download speeds. I'm fetching stuff from s3 into an ec2 instance in the same region and it is only doing 1.5MB/s (which is pretty much what I was getting to home from the same s3 bucket) [07:18] well, i guess i let that run [07:34] it'll be fine, there's more data where that came from [10:12] 200 GB/month?? I use 3500 from my home connection... [10:15] Nemo_bis, australia sucks [10:15] Italy has had a monopoly since few years ago... [10:15] and we're full of mountains :) [10:16] and and and and and [10:16] I've uploaded 1TB+ the latest month [10:16] from my home connection [10:16] what I dislike is that I mostly have nothing to download, 10 Mb/s full duplex are constantly half wasted [10:18] I wish I had internet like yours [10:18] i have backed up in 2005 efc.com ;) i may check drives and upload it somewhere [10:18] (efc - official site for TV SHow Earth final conflict) [12:08] SketchCow: If you have a little bit of time, could you create a "tabblo" rsync module? [12:26] uploading floss weekly podcast from 2006 [12:27] going to do this yearly since floss weekly is still making podcast [12:46] so ids will be floss_weekly_2006 to floss_weekly_2011 [13:22] SketchCow: http://archive.org/details/floss_weekly_2006 and http://archive.org/details/floss_weekly_2007 [13:23] first 4 episodes was gone of twit.tv but i found them [13:29] Counting files: 725 [13:29] Downloading finished! Yay! [13:29] Done. YAAAY! [13:29] Getting the size: 121G www.fileplanet.com/ [13:29] TARring! [13:29] cp: target `logs/' is not a directory [13:29] oops [13:30] I blame S[h]O[r]T and shaqfu [13:32] aaargh [13:32] i would be to blame [13:32] well, no problem actually [13:32] that is just a backup directory ;) [13:33] :) [13:33] okay, good [13:33] underscor: wait, what kind of range was that [13:33] that is way too huge [13:33] 222000-222999 [13:33] What to you mean too huge? [13:34] too big for one item at archive.org (since we want to be nice and keep them small) [13:34] in the table of ranges it says to please use ranges of 100-500 for that high IDs [13:34] :\ [13:35] would be awesome if you could split that into 100s chunks [13:35] Oh, this block didn't have that next to it iirc [13:35] Sorry [13:35] i'll update it to read "use 100" since i ran into problems in the 180k range too and it is easier to do small chunks than manually splitting [13:36] nah, sorry from my end. the table is not too obvious. the open ranges are not meant to be picked as a whole :} [13:36] I can resplit, but 120GB isn't too big on IA's end if you pass a size hint with s3 [13:36] Jason has stuff much larger ha [13:36] someone said we should aim smaller, iirc <50GB, the smaller the better [13:38] I suppose. Althoigh variable size pieces are a bad idea IMO [13:38] Whatever, it's your project and call :) [13:38] I'll repack when I have a few minutes [13:41] it's always good to be as nice as possible to IA, reduce friction and all~ [13:41] Oh, I know. Up to 200 or so GB isn't harmful/big deal as long as it makes sense as an item, however [15:56] SketchCow: http://archive.org/details/floss_weekly_2008 [16:22] so i ran my tabblo range x2 [16:22] should i run it a 3rd ? [16:22] seems like errors still [16:31] hey godane do you upload alot of gaming stuff to archive? [16:38] not really [17:20] http://archive.org/post/420547/teacher-forum-in-french-to-be-re-archived-urgently-please [18:29] http://25.media.tumblr.com/tumblr_m2wjow4q671rt3zyho1_500.gif [18:30] forums are a bitch to mirror [18:30] S[h]O[r]T: The errors are expected afaik [18:30] they're normal [18:30] they come up when a tabblo doesn't exist or was deleted [18:35] à¸à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹ à¸à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹à¹ [18:40] chronomex: (re: my s3 download) the problem is I wanted to drop the data from s3 before the end of the month, when my free tier usage ends. [18:41] Nemo_bis: "full fuplex"? I don't think it means what you think it means. I think you're looking for "symmetric" [18:45] Coderjoe: ah, suck [18:47] and s3cmd's sync command seems to be re-fetching everything, for some stupid reason, which costs in data usage. so now I'm pulling it to a volume within the same region on ec2 in order to rsync it. [18:48] unfortunately, it is taking longer to pull down to ec2 than I would like [18:59] @ underscor right. so i ran it twice per wiki. now what? [19:11] run the s3cmd? [19:11] Coderjoe: You can't multithread it? [19:12] isnt it supposed to upload all of them automatically? [19:14] do i just need to keep re running it until it doesnt do anything to make sure it gets everything? [19:15] underscor: not terribly easily, I don't think. I would have to dive down into the data directory [19:15] and manually run an s3cmd per directory there [19:16] User-agent: Googlebot [19:16] # Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl [19:16] Fuond in twitter's robots.txt [19:17] yeah, they require you do use an google account to connect to your site and use the webmaster tools [19:17] super shady [19:29] mmm [19:29] underscor: http://aws.amazon.com/code/Python/1684 [19:33] ooh [19:36] Coderjoe, dunno https://en.wikipedia.org/wiki/Full_duplex [19:36] hm maybe you're ight [20:42] an interesting online comic book http://kh43.com [21:26] How harsh is it on IA to link directly into tarview/ [21:26] ? [21:27] I want to index FP and have links directly to the files, but I also don't want to brutalize their machines [21:31] it's not that harsh [21:31] no more than someone downloading it [21:35] Sounds good, then; full steam ahead [21:35] make sure to use the permanent archive.org/download/foo/foo.tar/file method rather than a tarview url [21:37] 'course; trying to do this with tarview urls would be insane anyway [21:44] it likely is a little more difficult, since it needs to dig the file out, unless tarview and inside-tar download links use an index file that was made by the deriver or something. [21:48] whats tarview [21:49] http://archive.org/download/FileplanetFiles_00000-09999/00000-09999.tar/ [21:50] bsmith095: It's a view/file listing for items which are tars [22:23] SketchCow: can you get this item to start making ogg vorbis for the other items again? http://archive.org/details/floss_weekly_2008 [22:24] the titles was screwed up on upload for some reason [22:25] i check them with easytag and the title tag was fine but for some reason on archive.org it was junk [22:29] godane: id3 comes in two versions, perhaps that is it? [22:35] maybe [22:36] i just know easytag doesn't have the bad title name