#archiveteam 2012-02-09,Thu

↑back Search

Time Nickname Message
01:01 πŸ”— Coderjoe yipdw: wow. that article shows a gross misunderstanding as to how that bill was created
01:08 πŸ”— Coderjoe oh
01:08 πŸ”— Coderjoe i just noticed who the author was
03:38 πŸ”— godane i found out that there was a stripper at defcon 4
03:38 πŸ”— godane i'm surprise this was defcon.org though
03:53 πŸ”— dashcloud as part of one of the kickstarter campaigns I funded, they offered to make a game from an idea you had (very quick & dirty), so I put down saving web pages from the yahoo monster
04:34 πŸ”— underscor 13Ò£Ž4Ò£± 8ґ€Ò£€ 9Γ’Β’Β€11Ò£€ 12Ò£‡Ò‘€ 13Γ’Β Β„ 4ґ€8Γ’Β’Β€ 9Ғ€Ò‘€ 11Γ’Β’ΒΉ12Ò  13Ғ€Ò‘€ 4Γ’Β’Β€8Ò£€ 9Ò£€Ò£€
04:34 πŸ”— underscor 4Γ’Β Β‡8Γ’Β ΒΈ 9Ò  11Γ’Β Β£12Γ’Β Β€ 13Ò ‡Ò ¸ 4Γ’Β Β‡ 8Γ’Β Β±9Γ’Β Βƒ 11Ò £Ò ­ 12Γ’Β ΒΈ 4Ò £Ò ­ 8Γ’Β Β£9Γ’Β ΒΌ 11Ò ‡Ò ‡12Γ’Β Β‡
04:35 πŸ”— yipdw that needs to scroll right-to-left
04:36 πŸ”— underscor 13Γ―ΒΌΒ‘4Γ―Β½Β’8Γ―Β½Βƒ9ｈ11Γ―Β½Β‰12Γ―Β½Β–13Γ―Β½Β…4Γ―ΒΌΒ΄8Γ―Β½Β…9ａ11ｍ
04:37 πŸ”— arrith fabulous
04:46 πŸ”— underscor chronomex: op me por favor <3
04:50 πŸ”— haesken Hey so I'm interested in helping out with archiveteam, is there anything I can do? I have some experience writing web scrapers (python) if that is needed.
04:52 πŸ”— DFJustin for starters you can run the script at http://archiveteam.org/index.php?title=MobileMe
04:55 πŸ”— Aranje >_>
04:55 πŸ”— haesken ok, how much space should I allocate for it?
04:56 πŸ”— yipdw I think around 10-20 gigs per profile is fine
04:57 πŸ”— yipdw seesaw.sh uploads and deletes after every successful download, so you shouldn't have a lot of accumulation
05:00 πŸ”— haesken ok running :)
05:14 πŸ”— underscor :D
05:17 πŸ”— arrith maybe put that at the end or something :P
05:18 πŸ”— DFJustin 4,5o12,1h9,6 10,4g5,11o11,13d11,6 5,11m9,1y7,13 4,3e2,6y8,1e12,10s
05:19 πŸ”— yipdw I demand a useful topic
05:22 πŸ”— nitro2k01 yipdw: Set up UTF8 support, mmkay?
05:25 πŸ”— yipdw nitro2k01: I have UTF-8 support on
05:25 πŸ”— yipdw that does not make the topic content useful
05:25 πŸ”— yipdw well, it is now
05:25 πŸ”— yipdw or, put another way, being able to see "SketchCow for President" does not make the topic content useful
05:25 πŸ”— yipdw :P
05:27 πŸ”— underscor :D
07:19 πŸ”— Nemo_bis DFJustin, are you demanding a +c ?
07:20 πŸ”— DFJustin am I who's it what
07:22 πŸ”— DFJustin oh banning colour codes or something?
07:23 πŸ”— DFJustin nah I don't care
07:24 πŸ”— arrith tpb magnet link archive guy posted his script: http://pastebin.com/8RXXthXB
07:24 πŸ”— arrith "I went trough all the torrents, but as a logged out user. This may be the reason why I saw only 1.5m of torrents.
07:24 πŸ”— arrith I went trough the torrent description sites like
07:24 πŸ”— arrith It took the computer about 2 days. I can post the script somewhere if you want, it is in perl, it is not complicated at all.
07:24 πŸ”— arrith by adding 1 to the $i
07:24 πŸ”— arrith http://thepiratebay.se/torrent/$i
07:25 πŸ”— arrith "
07:25 πŸ”— arrith for reference: from the comments of https://thepiratebay.se/torrent/7016365/The_whole_Pirate_Bay_magnet_archive
07:25 πŸ”— arrith doing it again but logged in would be the first thing i'd think of
07:25 πŸ”— kennethre ugh perl
07:25 πŸ”— arrith kennethre: yeah :(
07:25 πŸ”— arrith readable and short enough to easily rewrite though
07:26 πŸ”— kennethre yeah
07:26 πŸ”— kennethre i wonder how fast i could distribute that
07:26 πŸ”— Nemo_bis DFJustin, I thought those colours were a form of protest
07:27 πŸ”— kennethre run the whole thing in 10 minutes or something ;)
07:28 πŸ”— Nemo_bis kennethre, I think you could download the entire mobileme in a few days :)
07:29 πŸ”— Nemo_bis how much disk space do you have? :D
07:29 πŸ”— kennethre Nemo_bis: hah, ephemeral so depends on where i'm uploading to ;)
07:30 πŸ”— Nemo_bis kennethre, how fast are you able to upload to batcave, usually?
07:30 πŸ”— Nemo_bis or to archive.org
07:30 πŸ”— kennethre i don't have metrics
07:30 πŸ”— Nemo_bis ah, I see underscor is downloading again
07:30 πŸ”— Nemo_bis ok
07:30 πŸ”— underscor yep
07:31 πŸ”— underscor Got 2 fresh 2TB drives
07:31 πŸ”— Nemo_bis :D
07:31 πŸ”— underscor Plus another 500GB
07:31 πŸ”— kennethre i got 180 GB uploaded in <20 hours (iirc)
07:31 πŸ”— Nemo_bis underscor, ah, so you're not using the IA machine?
07:31 πŸ”— underscor Liability
07:32 πŸ”— Nemo_bis oic
07:32 πŸ”— underscor If it's downloaded from IA IP space, then there are more legal options
07:32 πŸ”— underscor since IA is a "company"/library
07:32 πŸ”— underscor also, they like to keep an arms-length from us
07:32 πŸ”— underscor :D
07:32 πŸ”— underscor since we're kinda crazy and maniacal
07:33 πŸ”— Nemo_bis and uploading from the IA machines is not a problem?
07:33 πŸ”— kennethre what's being downloaded now?
07:33 πŸ”— Nemo_bis mobileme
07:33 πŸ”— kennethre Nemo_bis: where's the script?
07:33 πŸ”— underscor Nemo_bis: I dunno, I just want to play it safe
07:33 πŸ”— underscor I don't have a lot of space on my VM anyway
07:34 πŸ”— Nemo_bis kennethre, http://www.archiveteam.org/index.php?title=MobileMe
07:34 πŸ”— underscor https://github.com/ArchiveTeam/mobileme-grab/
07:34 πŸ”— Nemo_bis underscor, add it to the topic
07:34 πŸ”— Nemo_bis (the wiki page url)
07:34 πŸ”— Nemo_bis aww, /this/ is useful topic now
07:36 πŸ”— kennethre sweet
07:37 πŸ”— kennethre i guess i can spin up a few vms :)
07:37 πŸ”— kennethre can i use the same upload slots i used for splinder?
07:37 πŸ”— underscor No, there's a different one
07:37 πŸ”— underscor see the upload-finished.sh script
07:38 πŸ”— kennethre i'll need a slot immediately
07:38 πŸ”— underscor Use that, otherwise completion won't get reported to the tracker
07:38 πŸ”— underscor You don't need a slot provisioned
07:38 πŸ”— kennethre excellent
07:38 πŸ”— Nemo_bis so, we need a +77 % speed increase of the project to complete it
07:38 πŸ”— underscor Really??
07:39 πŸ”— kennethre well then
07:39 πŸ”— kennethre i'm on it :)
07:39 πŸ”— kennethre 200 instances should do
07:39 πŸ”— underscor kennethre: heroku again?
07:39 πŸ”— underscor :P
07:39 πŸ”— Nemo_bis underscor, yes and it was way worse before you restarted
07:39 πŸ”— kennethre underscor: but of course :
07:39 πŸ”— Coderjoe seesaw supposedly uploads and deletes completed profiles while downloading or something
07:39 πŸ”— Coderjoe (I haven't looked at it)
07:39 πŸ”— underscor kennethre: These are between a few k and 20GB, so hopefully that's not an issue
07:40 πŸ”— kennethre well i can't run a job for more than 24 hours
07:40 πŸ”— Nemo_bis one in ...?
07:40 πŸ”— Nemo_bis *once
07:41 πŸ”— kennethre so a few ginormous ones might need to be re-downloaded
07:41 πŸ”— underscor You don't think heroku will boot you off?
07:41 πŸ”— kennethre underscor: i work for heroku
07:41 πŸ”— underscor oh
07:41 πŸ”— kennethre :)
07:41 πŸ”— underscor hahahahahahaha
07:42 πŸ”— kennethre hehe
07:42 πŸ”— underscor niiice
07:42 πŸ”— kennethre how long has this been going on?
07:42 πŸ”— kennethre just noticed that the splinder readme had mobileme stuff in it
07:42 πŸ”— underscor "Hey, uh, our bandwidth usage spiked up to 10 gigabits last night. You don't know anything about it, do you?"
07:43 πŸ”— underscor Uh, long time ago
07:43 πŸ”— underscor Before Halloween
07:43 πŸ”— underscor But we didn't actively start until recently
07:43 πŸ”— underscor cf taiwan floods
07:43 πŸ”— Coderjoe thailand floods?
07:44 πŸ”— underscor er, whoops
07:44 πŸ”— underscor thanks
07:45 πŸ”— Coderjoe yeah, seesaw.sh downloads one user, uploads it to batcave, deletes it locally, notifies the tracker, and starts over
07:45 πŸ”— kennethre oh geeze
07:45 πŸ”— kennethre that's perfect
07:45 πŸ”— kennethre hmm
07:46 πŸ”— kennethre https://github.com/kennethreitz/heroku-splinder/blob/master/run.py
07:46 πŸ”— kennethre so here's what i did before
07:46 πŸ”— kennethre https://github.com/kennethreitz/heroku-splinder/blob/master/Makefile
07:46 πŸ”— kennethre i'll prob keep using streamer
07:47 πŸ”— underscor Why do that over seesaw?
07:47 πŸ”— Coderjoe different upload location, btw
07:47 πŸ”— underscor (also it's called dld-client now)
07:48 πŸ”— Coderjoe i hope the one drive with that rsync module doesn't get overloaded
07:49 πŸ”— kennethre because seesaw only does one at a time, right?
07:49 πŸ”— kennethre no concurrency
07:49 πŸ”— Coderjoe so when would your system actually upload anything?
07:50 πŸ”— kennethre every 2 minutes it ran the upload script
07:50 πŸ”— kennethre in the background
07:50 πŸ”— underscor none of the scripts do concurrency right now
07:50 πŸ”— underscor dld-client nor seesaw
07:50 πŸ”— underscor You just run multiples
07:50 πŸ”— kennethre dld-streamer.sh did ;(
07:50 πŸ”— underscor A single thread will max out a 100mbit connection though
07:50 πŸ”— kennethre ah they're nice and fast compared to splinder then
07:50 πŸ”— underscor It's akamai
07:51 πŸ”— Coderjoe splinder was all kinds of busted
07:51 πŸ”— kennethre shouldn't be an issue then :)
07:51 πŸ”— kennethre damn that's a ton of data
07:52 πŸ”— underscor yeah
07:52 πŸ”— underscor 211TB
07:52 πŸ”— underscor estimated
07:52 πŸ”— underscor :D
07:52 πŸ”— kennethre seesaw is awesome
07:52 πŸ”— kennethre that saves me so much trouble
07:52 πŸ”— Coderjoe anyone doing anything for imgur.com? lots of stuff uses that for hosting.
07:52 πŸ”— chronomex splinder was lots of tiny high-latency requests
07:53 πŸ”— kennethre yeah
07:53 πŸ”— kennethre i wasn't aware mobileme sites were on akamai
07:53 πŸ”— kennethre i know the real mobileme had a ton of issues
07:53 πŸ”— Coderjoe i wonder if the data i'm pulling down from my s3 bucket has been marked done
07:53 πŸ”— underscor We should do an imgur mirror
07:54 πŸ”— underscor https://www.youtube.com/watch?v=9MkbLQWHcXw
07:54 πŸ”— Aranje but they delete
07:54 πŸ”— Aranje :|
07:54 πŸ”— kennethre underscor: i bet there's a ton of illegal content
07:54 πŸ”— Coderjoe and i forget if i uploaded it to the batcave. (i suspect i did)
07:54 πŸ”— underscor kennethre: Archive first, (don't) delete later ;)
07:54 πŸ”— kennethre haha
07:55 πŸ”— Aranje Yeah basically :>
07:56 πŸ”— kennethre wow and get-wget-warc too
07:56 πŸ”— kennethre this is way better
07:56 πŸ”— kennethre easy as pie
07:56 πŸ”— kennethre is there a verbose mode for any of this stuff?
07:56 πŸ”— kennethre i pretty much can run and look at logs
07:56 πŸ”— underscor Does heroku give you basically a VM or something?
07:56 πŸ”— kennethre underscor: kinda, abstracted away
07:57 πŸ”— yipdw DYNOS!
07:57 πŸ”— yipdw :P
07:57 πŸ”— kennethre yipdw: :)
07:57 πŸ”— yipdw also yes there is a ton of illegal content on mobileme
07:57 πŸ”— yipdw I must have like gigs of new music
07:57 πŸ”— kennethre oh imeant imgur
07:57 πŸ”— yipdw most of which I am sure does not come with rights for redistribution
07:58 πŸ”— yipdw ohh
07:58 πŸ”— yipdw yeah that too :P
07:58 πŸ”— underscor <yipdw> I must have like gigs of new music
07:58 πŸ”— underscor hahaha
07:58 πŸ”— underscor interesting project would be to pull out all the music we find
07:58 πŸ”— yipdw I haven't actually checked all the WARCs
07:58 πŸ”— underscor or movies
07:58 πŸ”— yipdw but some of the biggest ones I've found contain either that or QuickTime movies
07:58 πŸ”— underscor I'm sure there's lots of movies too
07:59 πŸ”— yipdw home movies most likely
07:59 πŸ”— kennethre so ./seesaw.sh just exits when a user is done
07:59 πŸ”— yipdw oh, it uploads the user's data, deletes it, goes for another
07:59 πŸ”— kennethre so autonomous, excellent
08:00 πŸ”— kennethre i assume it repeats and repeats
08:00 πŸ”— yipdw yeah
08:00 πŸ”— yipdw it's too bad MobileMe is not hosted on e.g. S3
08:01 πŸ”— underscor Why?
08:01 πŸ”— yipdw because then Heroku dynos would have a hilarious speed advantage
08:01 πŸ”— kennethre yeah
08:01 πŸ”— yipdw (afaik they're on EC2 instances)
08:01 πŸ”— kennethre and free
08:01 πŸ”— yipdw heh
08:01 πŸ”— underscor oh, hahah
08:01 πŸ”— kennethre transfer
08:02 πŸ”— kennethre we'll get crazy speed anyway
08:02 πŸ”— kennethre i feel like upload will be the bottleneck
08:02 πŸ”— yipdw :P
08:02 πŸ”— yipdw I look forward to seeing kennethre bogart the dashboard
08:03 πŸ”— kennethre i really hope it happens :)
08:03 πŸ”— underscor not fair >:|
08:03 πŸ”— underscor I've been trying hard to get to the top spot
08:03 πŸ”— kennethre i'm not getting my hopes up
08:05 πŸ”— kennethre alright, here goes nothing.
08:05 πŸ”— yipdw how many processes?
08:05 πŸ”— kennethre haven't turned it on yet :)
08:05 πŸ”— yipdw oh
08:05 πŸ”— kennethre alright, 1
08:05 πŸ”— yipdw ONE
08:06 πŸ”— kennethre gah failure
08:06 πŸ”— kennethre gah it relies on wget
08:10 πŸ”— underscor That's what get-wget-warc does
08:10 πŸ”— underscor Oh wait, does get-wget-warc call wget?
08:10 πŸ”— underscor haha
08:10 πŸ”— kennethre yeah it does
08:10 πŸ”— kennethre i swapped it out with curl
08:10 πŸ”— kennethre alright i'm in action
08:11 πŸ”— kennethre i need to write an archive team buildpack
08:11 πŸ”— kennethre that has warc prebuilt
08:11 πŸ”— kennethre and rsync
08:11 πŸ”— yipdw the next wget release will have WARC writing capabilities present, but I'm not sure when that release will be
08:11 πŸ”— Nemo_bis kennethre, yes, I doubt you'll be able to upload everything immediately
08:11 πŸ”— Nemo_bis batcave is too slow for that
08:12 πŸ”— yipdw it's fast here
08:12 πŸ”— Nemo_bis and it doesn't have a lot of space
08:12 πŸ”— yipdw I regularly get at least 1 MB/s
08:12 πŸ”— Nemo_bis yipdw, how fast?
08:12 πŸ”— underscor /dev/vdc 16T 15T 1.5T 91% /2
08:12 πŸ”— underscor uh oh
08:12 πŸ”— Nemo_bis that's slow :-p
08:12 πŸ”— underscor D:
08:12 πŸ”— yipdw it's not too bad
08:12 πŸ”— underscor (that's the disk mobileme uploads are going to)
08:12 πŸ”— yipdw hmm
08:12 πŸ”— yipdw maybe those should be cleared off
08:12 πŸ”— Nemo_bis it is if you download at some 10-100 MB/s as kennethre probably will :p
08:13 πŸ”— yipdw it's not too bad so far
08:13 πŸ”— yipdw he has so far downloaded < 1 MB
08:13 πŸ”— yipdw :P
08:13 πŸ”— kennethre haha
08:13 πŸ”— kennethre building now
08:13 πŸ”— yipdw underscor: where exactly is the mobileme data going to end up?
08:13 πŸ”— yipdw 200 TB is quite a tall order to provision
08:14 πŸ”— underscor SketchCow is talking with archive folks
08:14 πŸ”— underscor I think the current plan is to stick things in temporary items on the cluster
08:14 πŸ”— underscor then slowly "download" them and reorganize and then reupload
08:14 πŸ”— ersi Where's the dash?
08:15 πŸ”— underscor Since they don't have 200TB of disks just laying around
08:15 πŸ”— underscor hahaha
08:15 πŸ”— underscor ersi: The dash?
08:15 πŸ”— ersi +board
08:15 πŸ”— underscor memac.heroku
08:15 πŸ”— yipdw .com
08:16 πŸ”— ersi neat
08:18 πŸ”— ersi If I'm running dld-client.sh, does stats get reported to the tracker?
08:18 πŸ”— yipdw yeah
08:19 πŸ”— kennethre alright
08:19 πŸ”— kennethre i think i'm in action
08:20 πŸ”— kennethre hroku scale scrape=100
08:20 πŸ”— underscor :D
08:21 πŸ”— underscor do you get free resources?
08:21 πŸ”— kennethre yeah, my bill is $0 every month
08:21 πŸ”— kennethre no matter what i use
08:21 πŸ”— kennethre obviously i shouldn't do anything crazy
08:21 πŸ”— kennethre hopefully this won't be too intensive
08:23 πŸ”— underscor TThat's cool
08:24 πŸ”— kennethre wow it's flying
08:24 πŸ”— kennethre this worries me
08:24 πŸ”— kennethre in the right on the dashboard
08:25 πŸ”— kennethre those are already-uploaded, right?
08:25 πŸ”— underscor No, those are completed
08:26 πŸ”— underscor They don't get reshown when they upload
08:26 πŸ”— underscor Are you using seesaw?
08:26 πŸ”— kennethre i am
08:26 πŸ”— kennethre you weren't kidding when you said it'd exhaust the connection
08:26 πŸ”— kennethre damn
08:26 πŸ”— yipdw aaand there it goes
08:27 πŸ”— underscor ho' shi', boy!
08:27 πŸ”— underscor that's crazy
08:27 πŸ”— underscor I want to see cacti/mrtg graphs for the ec2 instances
08:27 πŸ”— underscor :D
08:27 πŸ”— kennethre hopefully this will help you keep up :)
08:27 πŸ”— kennethre this kind of saturation worries me
08:27 πŸ”— kennethre i need to check a few things
08:27 πŸ”— underscor haha
08:27 πŸ”— kennethre i wasn't expecting it to go nearly this fast
08:28 πŸ”— underscor "Uh, Kenneth. You broke EVERYTHING!"
08:28 πŸ”— kennethre nah not at all
08:28 πŸ”— underscor haha
08:28 πŸ”— kennethre just abusing resources ;)
08:28 πŸ”— underscor "slightly"
08:29 πŸ”— kennethre can someone tell me how much my user has uploaded so far?
08:29 πŸ”— kennethre is there an api for that?
08:29 πŸ”— kennethre just trying to gauge the current speed
08:29 πŸ”— underscor kennethre: I can, one sec
08:29 πŸ”— underscor No API, but I can du your module
08:30 πŸ”— kennethre it's been up for ~8 minutes
08:31 πŸ”— underscor du: cannot read directory `kenneth/m': Permission denied
08:31 πŸ”— underscor Oh, nevermind then
08:31 πŸ”— kennethre :(
08:31 πŸ”— underscor I don't have sudo rights on SketchCow's box
08:32 πŸ”— underscor kennethre: Upload your files with global read and exec
08:32 πŸ”— underscor :P
08:32 πŸ”— underscor Everyone elses are 755
08:32 πŸ”— kennethre underscor: hey i'm just running the script :)
08:32 πŸ”— underscor Yours are 700 :P
08:33 πŸ”— kennethre well a bit late for that now ;)
08:33 πŸ”— underscor Heroku bug?
08:33 πŸ”— underscor Well
08:33 πŸ”— underscor Not bug
08:33 πŸ”— underscor but default
08:33 πŸ”— kennethre prob just a configuration difference
08:33 πŸ”— underscor yeah
08:34 πŸ”— underscor Hmm
08:34 πŸ”— underscor Oh, hey, mobileme isn't on /2
08:34 πŸ”— underscor Whoops
08:34 πŸ”— kennethre haha
08:34 πŸ”— underscor /dev/vdd 11T 8.4T 2.2T 80% /3
08:34 πŸ”— underscor So, little more breathing room
08:34 πŸ”— kennethre if you're curious, here's what i'm doing https://github.com/kennethreitz/heroku-mobileme
08:35 πŸ”— underscor Wow, you're killing the dashboard
08:37 πŸ”— kennethre :)
08:37 πŸ”— kennethre this is far more fun than you think.
08:37 πŸ”— kennethre i could scale it up way higher
08:37 πŸ”— kennethre but i think this'll do for now
08:38 πŸ”— ersi underscor: probably another umask for his user, either on the upload box or download boxes
08:39 πŸ”— underscor kennethre: /me waits for akamai to call up heroku
08:39 πŸ”— underscor "Hey, uh, could you stop ddosing us?"
08:39 πŸ”— underscor :D
08:39 πŸ”— kennethre that would deeply suck
08:39 πŸ”— kennethre i think they'd call ec2 first
08:39 πŸ”— ersi I bet Apple will get a nice bill from Akamai
08:39 πŸ”— underscor hehe
08:39 πŸ”— underscor I'm sure that they're used to much higher load
08:39 πŸ”— ersi Monetary Denial of Service
08:39 πŸ”— kennethre HAH
08:40 πŸ”— ersi yeah, I bet they see hueg spikes occationally
08:40 πŸ”— kennethre have you seen their room?
08:40 πŸ”— underscor Batcave is receiving at 10MB/s
08:40 πŸ”— ersi but yeah, I bet kennethre is throwing them a punch haha
08:40 πŸ”— underscor (so 80mbps)
08:40 πŸ”— kennethre http://www.youtube.com/watch?v=kiZgsBUEJdA
08:40 πŸ”— kennethre (Inside Akamai's war room)
08:41 πŸ”— kennethre i think it's pretty obvious we're not ddosing
08:41 πŸ”— underscor Oh man
08:41 πŸ”— underscor That's cool!
08:41 πŸ”— underscor I want to visit that room
08:41 πŸ”— kennethre it's not like we're slow lorising them :)
08:41 πŸ”— underscor hehe
08:41 πŸ”— ersi oh man, youtube commercials for the fail
08:41 πŸ”— underscor Oh, wow
08:42 πŸ”— underscor It's Ahhh kamai
08:42 πŸ”— underscor not aaaaa kamai
08:42 πŸ”— underscor I've been saying it wrong for a while
08:42 πŸ”— ersi start spelling it out instead, a k a m a i
08:43 πŸ”— ersi can't go wrong with that
08:43 πŸ”— underscor :D
08:44 πŸ”— underscor Is anyone else pausing it so you can read everything on each display?
08:44 πŸ”— underscor haha
08:44 πŸ”— underscor http://www.youtube.com/watch?feature=player_detailpage&v=kiZgsBUEJdA#t=26s
08:44 πŸ”— underscor 2.6 terabits/sec?!?!?!?!
08:45 πŸ”— kennethre hahaha
08:45 πŸ”— underscor That's SO COOL
08:45 πŸ”— kennethre they power like 70% of the internet
08:45 πŸ”— underscor haha
08:45 πŸ”— kennethre 12.7 million requests per second
08:45 πŸ”— kennethre wow.
08:46 πŸ”— underscor kennethre: Batcave intake has jumped from ~50-80mbps to ~184-220mbps since you started
08:46 πŸ”— yipdw kennethre: https://github.com/kennethreitz/heroku-mobileme/commit/3f8ce58702cdbdf5dca3f48e4aa112b111d2a3a1 <-- hah what
08:46 πŸ”— kennethre yipdw: yeah, i'm just telling heroku it's a python app
08:47 πŸ”— kennethre yipdw: since i'm too lazy to do a custom buildpack
08:47 πŸ”— yipdw heh
08:47 πŸ”— kennethre underscor: that's freaking awesome ;)
08:47 πŸ”— underscor haha
08:47 πŸ”— underscor I wish I had sudo
08:47 πŸ”— kennethre underscor: what kind of connection does that thing have?
08:47 πŸ”— underscor Then I could run iftop
08:48 πŸ”— ersi Heh, first user I got; - Running wget --mirror (at least 16520 files)...
08:48 πŸ”— ersi 400MB downloaded atm
08:48 πŸ”— underscor kennethre: Gigabit copper
08:49 πŸ”— kennethre underscor: strait to a backbone? :P
08:49 πŸ”— underscor No
08:49 πŸ”— kennethre i'm joking
08:49 πŸ”— yipdw I have a whole 1 mbps up
08:49 πŸ”— kennethre so assuming i can keep this up, we'll hit the deadline?
08:49 πŸ”— underscor VM (batcave) -> host (gbit copper) -> switch -> switch -> router -> router -> internet
08:50 πŸ”— underscor kennethre: http://ia700000.us.archive.org:8088/mrtg/
08:50 πŸ”— underscor If you want to watch some stats
08:50 πŸ”— kennethre ah excellent
08:50 πŸ”— kennethre thanks :)
08:50 πŸ”— yipdw haha
08:50 πŸ”— yipdw I like the spike
08:50 πŸ”— underscor The network graphs are in bytes, fyi
08:50 πŸ”— kennethre i'm really tempted to spin up another 100
08:51 πŸ”— underscor This host is 128GB ram
08:51 πŸ”— * underscor wishes he had that to play with
08:51 πŸ”— underscor kennethre: do eet!
08:51 πŸ”— underscor lol
08:51 πŸ”— kennethre err
08:51 πŸ”— kennethre is this a good idea
08:51 πŸ”— kennethre hmm
08:52 πŸ”— underscor Wow, the disk is filling at about 20MB/s
08:52 πŸ”— underscor 2271450 MB left
08:52 πŸ”— kennethre heroku scale scrape2=100
08:52 πŸ”— underscor 31 hours until the drive is full
08:52 πŸ”— kennethre will take about two minutes for 100 machines to build rsync and wget
08:52 πŸ”— underscor :D
08:52 πŸ”— kennethre i hope there's another drive
08:52 πŸ”— underscor no, SketchCow will have to do something
08:53 πŸ”— underscor Either handle it himself or give me access
08:53 πŸ”— underscor because he's in finland
08:53 πŸ”— yipdw this makes me appreciate the mind-boggling scale of EC2 more
08:53 πŸ”— kennethre ah
08:53 πŸ”— kennethre documentering?
08:53 πŸ”— underscor yeah
08:53 πŸ”— underscor and attending some hackfest or something
08:53 πŸ”— underscor helsinki hacklab iirc
08:53 πŸ”— kennethre awesome
08:54 πŸ”— underscor who wants to take bets as to when kennethre starts getting graphed?
08:54 πŸ”— underscor haha
08:54 πŸ”— yipdw 30 hours
08:54 πŸ”— yipdw 15
08:54 πŸ”— yipdw well hmm
08:54 πŸ”— kennethre maybe i should spin up another 200
08:54 πŸ”— kennethre and try to stay up until it happens
08:54 πŸ”— yipdw wtf
08:55 πŸ”— kennethre lol
08:55 πŸ”— yipdw is that 200 dynos?
08:55 πŸ”— kennethre yes
08:55 πŸ”— yipdw what the
08:55 πŸ”— yipdw I can't even price that on your site
08:55 πŸ”— ersi haha
08:55 πŸ”— kennethre $0.05/hour
08:56 πŸ”— kennethre per dyno
08:56 πŸ”— yipdw oh jeez
08:56 πŸ”— kennethre so currently $10 an hour
08:56 πŸ”— kennethre considering what i've downloaded so far, that's a hell of a deal
08:56 πŸ”— ersi Alright, if we get a "Heroku is out of business" news from somewhere - we know who to blame
08:56 πŸ”— ersi :P
08:56 πŸ”— underscor haha
08:56 πŸ”— underscor That's actually a really good deal
08:56 πŸ”— kennethre well we're currently running 1.1 million apps
08:57 πŸ”— underscor If we ever needed to do a really fast "oh shit" backup
08:57 πŸ”— kennethre yeah if that ever happens, call me :)
08:57 πŸ”— kennethre lol
08:57 πŸ”— underscor What's stopping someone from using that as a DOS manuver?
08:57 πŸ”— underscor maneuver*
08:57 πŸ”— kennethre nothing really
08:57 πŸ”— yipdw threat of being fired
08:57 πŸ”— yipdw :P
08:57 πŸ”— kennethre haha
08:58 πŸ”— ersi You could fire up a bunch of pre-built "attack VMs" and up on any platform
08:58 πŸ”— ersi Rackspace, Linode, EC2
08:58 πŸ”— yipdw I also think we probably DDoSed Splinder
08:58 πŸ”— kennethre heroku makes it particularly easy since we're app-level
08:58 πŸ”— ersi yeah, to hell and back
08:58 πŸ”— kennethre not system-level
08:58 πŸ”— ersi kennethre: true, but making a pre built image isn't hard :)
08:58 πŸ”— ersi even though I bet it's harder than whatever you did
08:58 πŸ”— kennethre ersi: takes longer than 5 minutes
08:58 πŸ”— yipdw also, YouTube shutting down in 24 hours
08:58 πŸ”— ersi considering time and details
08:58 πŸ”— kennethre lol
08:58 πŸ”— ersi yeah
08:58 πŸ”— ersi yipdw: wat
08:59 πŸ”— kennethre but yeah an ami for this would be awesome
08:59 πŸ”— yipdw no just kidding
08:59 πŸ”— kennethre one of the XXLs
08:59 πŸ”— yipdw somehow I don't think even all of EC2 could save YouTube in 24 hours
09:00 πŸ”— underscor haha
09:01 πŸ”— kennethre it's possible
09:02 πŸ”— kennethre my dad wil soon have access to the storage
09:02 πŸ”— kennethre *will
09:02 πŸ”— kennethre he's going to go work on a system with 230PB of storage :)
09:02 πŸ”— yipdw wow
09:02 πŸ”— kennethre he'll be working to get them 1TB/s io
09:02 πŸ”— yipdw imaging system?
09:02 πŸ”— kennethre (not a typo)
09:02 πŸ”— yipdw LHC?
09:02 πŸ”— kennethre hah
09:02 πŸ”— kennethre let me find it
09:03 πŸ”— underscor kennethre: http://tracker.archive.org/df.html
09:03 πŸ”— underscor that's the disk space remaining on batcave
09:04 πŸ”— kennethre O_o
09:04 πŸ”— underscor (live updating)
09:04 πŸ”— yipdw title that COUNTDOWN TO DESTRUCTION
09:04 πŸ”— yipdw lots of programs tend to go haywire when there's no more space left
09:04 πŸ”— kennethre http://www.informationweek.com/news/government/enterprise-architecture/231900554
09:05 πŸ”— underscor yipdw: refresh
09:05 πŸ”— kennethre that's the computer he'll be working on
09:05 πŸ”— yipdw underscor: I approve
09:05 πŸ”— yipdw kennethre: ooh awesome
09:05 πŸ”— yipdw 960 Tesla M2090 GPUs
09:05 πŸ”— yipdw :O
09:06 πŸ”— kennethre yipdw: that's the old one that's being replaced ;)
09:06 πŸ”— yipdw oh wait
09:06 πŸ”— kennethre oh wait nvm
09:06 πŸ”— kennethre you're right
09:06 πŸ”— yipdw oh
09:06 πŸ”— underscor kennethre: woah, that's cool
09:06 πŸ”— yipdw yeah, that's phase 1
09:06 πŸ”— kennethre "outfitting the machine with as many as 18,000 Tesla GPUs"
09:06 πŸ”— yipdw phase 2 appearently is "outfitting the machine with as many as 18,000 Tesla GPUs"
09:06 πŸ”— yipdw hah
09:06 πŸ”— kennethre lol
09:06 πŸ”— kennethre it's going to be awesome
09:07 πŸ”— yipdw I've been learning CUDA programming in my spare time and even the little toys I've run on my GTX 560 Ti blow me away
09:07 πŸ”— kennethre ah only 55PB of storage
09:07 πŸ”— yipdw scaling something to 18,000 GPUs must either be awesome or hell
09:07 πŸ”— kennethre i can't imagine
09:07 πŸ”— kennethre i have a hard enough time understanding gevent :)
09:07 πŸ”— yipdw I imagine you're going to hit bandwidth issues with main memory <-> GPU memory transfers
09:08 πŸ”— underscor yipdw: There, I added some graphics too
09:08 πŸ”— yipdw what the
09:08 πŸ”— yipdw man, that is declining fast
09:09 πŸ”— underscor yeah, 26MB/s
09:09 πŸ”— underscor Oh, wait
09:09 πŸ”— underscor 36MB/s now
09:09 πŸ”— underscor kennethre: you're insane :D
09:10 πŸ”— kennethre I do what i can :)
09:10 πŸ”— ersi web.me.com/venstrehaandsarbejde is taken over 1GB now >_>
09:10 πŸ”— underscor I should probably email SketchCow
09:10 πŸ”— yipdw Subject: NEED NEW DRIVE NOW
09:10 πŸ”— ersi It's 11:11 AM over in Finland right now btw
09:11 πŸ”— ersi e
09:11 πŸ”— ersi so should be awak
09:11 πŸ”— underscor oh, cool
09:11 πŸ”— underscor not on skype though
09:11 πŸ”— underscor Overall Top RX Speed: 118128.65 KB/s
09:11 πŸ”— underscor ^ kennethre
09:11 πŸ”— underscor 118 megabytes!!!!!!!
09:11 πŸ”— kennethre oh god
09:12 πŸ”— underscor That's actually maxing the disk controller
09:12 πŸ”— underscor :D
09:12 πŸ”— yipdw I just noticed he checked in something like 8 gigabytes of data
09:12 πŸ”— yipdw maybe faster than 15 hours
09:12 πŸ”— yipdw it's been what, 30 minutes?
09:12 πŸ”— underscor hahaha
09:12 πŸ”— kennethre 20
09:12 πŸ”— kennethre 52 for the first half, 20 for the second
09:12 πŸ”— yipdw hah
09:12 πŸ”— alard Something else about mobileme: It's not really urgent, but I've just updated the wget version. If you have the time, please run git clone and get-wget-warc to get the latest version. There was a problem downloading files >2GB to a WARC file, which should be fixed in the new version. (Not that there are many users with such large files.)
09:12 πŸ”— underscor Usage at 12:55AM GMT was 2267348
09:13 πŸ”— yipdw haha
09:13 πŸ”— underscor Er, disk free
09:13 πŸ”— yipdw more build times ahead
09:13 πŸ”— underscor So
09:14 πŸ”— underscor 29GB in 19 minutes
09:14 πŸ”— underscor (not all kennethre, but mostly)
09:14 πŸ”— underscor Wait, that doesn't mke sense
09:15 πŸ”— underscor Oh, wait, yeah
09:15 πŸ”— underscor Nevermind
09:15 πŸ”— underscor I was thinking that was 29GB in 19 seconds and going "wait a sec... lol"
09:15 πŸ”— underscor So 1528MB/min
09:15 πŸ”— kennethre haha
09:15 πŸ”— kennethre i was doing my math at 118 a minute
09:15 πŸ”— kennethre not second..
09:16 πŸ”— underscor 1462 minutes until dday
09:16 πŸ”— ersi alard: So.. files over 2GB will be capped? Is it fixable/noticable?
09:16 πŸ”— underscor So roughly 24 hours at the current rate
09:16 πŸ”— alard No, they won't be capped. wget will just crash.
09:16 πŸ”— underscor and it's accelerating
09:16 πŸ”— ersi alard: hah, awesome. Well, I guess that's noticable
09:16 πŸ”— underscor Ah, that explains the one user that kept segfaulting
09:17 πŸ”— alard Yes, that's probably the reason.
09:17 πŸ”— alard I discovered the problem with some users on umich.edu.
09:17 πŸ”— underscor kennethre: Idling around 49MB/s currently
09:17 πŸ”— kennethre weak
09:17 πŸ”— kennethre :)
09:18 πŸ”— underscor If that gets close to 0, can you call off your army?
09:18 πŸ”— underscor haha
09:18 πŸ”— kennethre unfortunately i'm having some processes being killed
09:18 πŸ”— underscor Aww, why?
09:18 πŸ”— ersi Hm, does doing this with warc mean that we get the data * 2?
09:18 πŸ”— kennethre memory quota
09:18 πŸ”— kennethre exceeding 512MB
09:18 πŸ”— underscor oh, yeah, they do that
09:18 πŸ”— kennethre rare though
09:18 πŸ”— underscor yeah
09:18 πŸ”— underscor usually just really big users
09:19 πŸ”— ersi whoa, wget is using like no memory at all these days :o
09:19 πŸ”— underscor yeah, thank alard
09:19 πŸ”— underscor (I think it was him?)
09:19 πŸ”— ersi alard: you the man
09:19 πŸ”— ersi Or who ever triaged that --mirror bug :)
09:19 πŸ”— underscor there were some things that were not free'd
09:20 πŸ”— ersi yeah, was like two three memleaks or something like that if I recall correctly
09:24 πŸ”— yipdw huh
09:24 πŸ”— yipdw I might just be crazy but I think a terabyte or two has been downloaded in the past half hour
09:24 πŸ”— yipdw from mobileme
09:24 πŸ”— yipdw at least I remember the total being 11000-something GB
09:24 πŸ”— yipdw and now it's 13036
09:30 πŸ”— kennethre underscor: hahahaha
09:30 πŸ”— kennethre underscor: the countdown to destruction
09:34 πŸ”— underscor wait
09:35 πŸ”— underscor efnet has a channel limit?????
09:35 πŸ”— underscor * #retroshare :You have joined too many channels
09:35 πŸ”— ersi yeah, all networks have a channel limit
09:35 πŸ”— underscor oh, hah
09:36 πŸ”— underscor I guess I've just never hit it before
09:36 πŸ”— ersi It's usually 20
09:36 πŸ”— ersi but could be anything
09:36 πŸ”— chronomex 20 is far too low
09:36 πŸ”— ersi jebus, 2GB on this user
09:37 πŸ”— ersi Hurr.. I'm inserting 48*3600*24*31 rows into MySQL
09:42 πŸ”— kennethre i really want to spin up more
09:42 πŸ”— kennethre i'll wait until the disk problem goes away though
09:43 πŸ”— underscor haha
09:47 πŸ”— underscor kennethre: http://ia700000.us.archive.org:8088/mrtg/iowait.html hehe
09:58 πŸ”— chronomex jeepers, what are you doing
10:03 πŸ”— kennethre proving that the cloud goes to 11
10:04 πŸ”— Nemo_bis FreeNode doesn't seem to have any limit
10:04 πŸ”— Nemo_bis I once joined some hundreds (thousands?) channels
10:05 πŸ”— chronomex and how did irssi like that
10:07 πŸ”— chronomex not well, I expect
10:07 πŸ”— ersi irssi handles that fine
10:07 πŸ”— ersi I've spammed 4000 PM's at irssi just fine
10:08 πŸ”— ersi ie unique windows
10:08 πŸ”— chronomex but 4000 active channels?
10:08 πŸ”— ersi same thing more or less
10:08 πŸ”— ersi mIRC crashes if you do the same thing anyhow
10:09 πŸ”— Nemo_bis irssi didn't even notice I was doing something strange
10:09 πŸ”— chronomex I am shocked, shocked that mirc doesn't like that
10:09 πŸ”— ersi sent like 500 PMs to a mIRC instace and it b0rked
10:09 πŸ”— Nemo_bis you only have to set a timer because there's a flood limit on the network
10:09 πŸ”— ersi Yeah, me neither. But when I was doing what I was doing..
10:09 πŸ”— ersi yeah
10:09 πŸ”— ersi you'll hit the buffer limit very quickly if you try to join them all at once
10:09 πŸ”— Nemo_bis yep
10:10 πŸ”— Nemo_bis and also get kicked by some overzealous staff perhaps
10:10 πŸ”— Nemo_bis because they have all sorts of smart reporting bots
10:11 πŸ”— kennethre think i'm going to bed
10:11 πŸ”— kennethre should i spin up more instances first?
10:11 πŸ”— kennethre eh i might as well stay up
10:15 πŸ”— Nemo_bis I want moar
10:15 πŸ”— Nemo_bis kennethre, how's your RTT to batcave?
10:16 πŸ”— kennethre good question
10:17 πŸ”— kennethre wow, no ping
10:17 πŸ”— kennethre easier said than done :)
10:17 πŸ”— Nemo_bis rtt min/avg/max/mdev = 192.448/246.892/286.526/27.767 ms
10:17 πŸ”— Nemo_bis twice as faster than usual from here, who knows why
10:18 πŸ”— kennethre because we're awesome
10:18 πŸ”— kennethre and cloud scale
10:19 πŸ”— kennethre alright, don't kill me
10:19 πŸ”— Nemo_bis and still the traceroute seems more silly than usual
10:19 πŸ”— kennethre i'm feeling ballsy tonight
10:19 πŸ”— kennethre i'm about to get 50% faster
10:20 πŸ”— kennethre then sleeping
10:21 πŸ”— chronomex sleep 50% faster?
10:21 πŸ”— chronomex sounds good to me
10:21 πŸ”— chronomex I'd like to sleep less
10:22 πŸ”— kennethre me too
10:22 πŸ”— kennethre i meant downloads
10:22 πŸ”— kennethre :P
10:32 πŸ”— kennethre BAM
10:32 πŸ”— SketchCow Morning
10:32 πŸ”— kennethre i made dashboard
10:32 πŸ”— SketchCow kennethre: I assume you're slamming the moblieme download.
10:33 πŸ”— kennethre SketchCow: indeed Ҁ” 387GB in 2 hours ;)
10:33 πŸ”— SketchCow Where's the dashboard?
10:33 πŸ”— kennethre http://memac.heroku.com/
10:36 πŸ”— Nemo_bis 26 days to complete it at this speed (theoric)
10:41 πŸ”— SketchCow Is the system you're using marking things as "done" so I can throw them into archive.org?
10:41 πŸ”— alard SketchCow: I was just going to write: 'Let me know if you need help with the processing of the mobileme rsync uploads.'
10:42 πŸ”— alard There's a list of users that have been uploaded to the new mobileme rsync module.
10:43 πŸ”— alard If that's useful, I could write you a little script that monitors that log and moves users from the upload directory to another place when they're done. Maybe making batches of 100GB each, which you can then move somewhere else when you're ready.
10:43 πŸ”— alard Or I could just give you a pointer to the log.
10:48 πŸ”— SketchCow I need help, let's put it that way.
10:49 πŸ”— SketchCow Because if we really are slamming in 387gb in two hours, I really need to start pushing that stuff out to archive.org proper ASAP.
10:49 πŸ”— SketchCow A script written by the very guy who wrote the script that is putting in the data in the first place seems the superior solution.
10:51 πŸ”— underscor haha
10:51 πŸ”— alard What happens is this: 1. The upload script rsyncs an user to batcave/mobileme/UPLOADER/U/US/USE/USER/
10:51 πŸ”— alard 2. The upload script sends the message 'USER uploaded by UPLOADER' to the tracker.
10:52 πŸ”— alard Now, somehow, there should be something that monitors that log and processes the uploaded users, moves them from the rsync path to somewhere else.
10:52 πŸ”— SketchCow Yes.
10:53 πŸ”— SketchCow Do something that is run from mobileme.
10:53 πŸ”— SketchCow It's OK to write a hook and let me write the final uploader.
10:53 πŸ”— SketchCow In many cases, we're probably going to .zip up the user
10:54 πŸ”— alard Do you want one big stream of individual users? Or do you want batches?
11:01 πŸ”— SketchCow Let's talk about this.
11:01 πŸ”— SketchCow I keep getting pulled into things, so I might disappear, but...
11:02 πŸ”— SketchCow I need to add sets of items to archive.org. It would be better for us to pop together 200gb sets of persons into archive.org's sets.
11:02 πŸ”— SketchCow You know, roughly 200gb. Could be 150gb if needed, say. But don't want to add billions of items into their DB.
11:04 πŸ”— Schbirid SketchCow: do you cover http://en.wikipedia.org/wiki/UltraCade_Technologies in ARCADE?
11:04 πŸ”— SketchCow My schedule. I'm in Finland today, at a newspaper, actually, watching my fiancee' talk to 50 reporters, management and others about the future of content.
11:05 πŸ”— SketchCow Schbirid: Wrong channel
11:05 πŸ”— Schbirid oops
11:05 πŸ”— Schbirid #jsmess ? ;D
11:05 πŸ”— SketchCow I'm online, and I can talk about archiveteam stuff but I suspect we'll be onto a new archive.
11:05 πŸ”— SketchCow Just e-mail me jason@textfiles.com with "I herd u makin movie plz include xxxxx thnks" like the 4,000 other people have.
11:06 πŸ”— Schbirid heh
11:06 πŸ”— Schbirid maybe!
11:06 πŸ”— SketchCow Let's try my last line again.
11:06 πŸ”— Schbirid i am researching the quake arcade tournament cabinet at the moment
11:06 πŸ”— ersi channel as in communications channel, not IRC channel
11:06 πŸ”— Schbirid aye
11:06 πŸ”— Schbirid i am german, is that excuse enough?
11:06 πŸ”— SketchCow "I'm online, and I can tak for now, but I am accompanying her tour today, and so I might be puled away."
11:07 πŸ”— ersi Gesundheit
11:07 πŸ”— SketchCow I owe one of my jobs an article tonight, that will take my attention.
11:07 πŸ”— ersi MobileMe feels darn large, and I've only done 10-30GB as of yet :/
11:08 πŸ”— SketchCow But after this, I can really concntrate on the data move, so alard, that would help a lot.
11:08 πŸ”— Schbirid i kinda want to try archiving http://home.swipnet.se/
11:08 πŸ”— SketchCow Also, batcave is going away, moving to a new machine.
11:08 πŸ”— Schbirid seems straightforward, it is URLs like ~w-25876/
11:08 πŸ”— Schbirid err http://home.swipnet.se/~w-25876/
11:08 πŸ”— SketchCow I want to move stuff into archive.org rather than the new machine.
11:08 πŸ”— Schbirid and you just increment the number
11:08 πŸ”— SketchCow So that's the news.
11:08 πŸ”— Schbirid not sure if the character too
11:09 πŸ”— Schbirid ah, google says there are normal urls too, as in /somestring/
11:09 πŸ”— ersi Schbirid: I'd be on for that, considering being a Swede and all
11:09 πŸ”— alard SketchCow: Let's move this discussion to #memac
11:10 πŸ”— ersi Schbirid: I'd also like to fetch all of http://medlem.spray.se/member/
11:10 πŸ”— ersi one of the old "geocities of Sweden"
11:10 πŸ”— Schbirid :)
11:10 πŸ”— SketchCow alard: Excellent idea
11:10 πŸ”— SketchCow ersi: Is it in danger of disappearing?
11:11 πŸ”— ersi Well, it's owned by a bunch of morons
11:11 πŸ”— ersi so they're of the flippity flop kind of people
11:11 πŸ”— ersi but they've managed to stay online as of yet, dunno if all of their legacy material is though
11:12 πŸ”— ersi I'll start looking into it though :)
11:13 πŸ”— Schbirid it is amazing how information dense those "oldschool websites" often are
11:13 πŸ”— Schbirid not some polished empty fluff
11:13 πŸ”— ersi mostly just text/images/sounds, ie content :)
11:14 πŸ”— Schbirid yeah
11:16 πŸ”— Schbirid "If you cannot see the navigation window on the left, or the animation on top, please download Macromedia's Flash plugin."
11:35 πŸ”— SketchCow I am all for "warning", is all
11:35 πŸ”— SketchCow But we are downloading a LOT of data right now.
11:35 πŸ”— SketchCow Mobile Me... I mean, ow, that thing is huge.
11:44 πŸ”— ersi Yeah, of course. That comes first
11:44 πŸ”— ersi Doesn't hurt to scout around and plan a little though :)
12:06 πŸ”— SketchCow Back one a gain.
12:17 πŸ”— SketchCow Now I'm in this beautiful meeting and symphony/performing space in Lahti, Finland.
12:51 πŸ”— ersi sweet
13:03 πŸ”— SketchCow kennethre: Drop to half of what you're doing.
13:03 πŸ”— SketchCow You're killing that machine and at current rates I will not have the push out to archive.org done before you've filled the machine. :)
13:04 πŸ”— SketchCow We're too good at acquiring data and I need to do one more round of pushing files out.
13:07 πŸ”— ersi I think he might have gone to sleep
13:08 πŸ”— ersi He was considering it two and a half hours ago at least
13:15 πŸ”— SketchCow YEah
13:15 πŸ”— SketchCow We're gonna get raped
13:15 πŸ”— SketchCow I see where this is going
13:15 πŸ”— SketchCow The machine is actually slowing down and not working well now.
13:15 πŸ”— ersi You could shut down the tracker temporarily
13:15 πŸ”— ersi that'll make him stop fetching jobs
13:15 πŸ”— SketchCow I could shoot the dog, that'll make him stop eating
13:16 πŸ”— ersi Indeed
13:16 πŸ”— ersi I was merely suggesting that maybe it's better to die prematurely with disk space left, than having everything go until full stop crash and burn
13:16 πŸ”— Nemo_bis but why does network decrease and CPU increase at the same time
13:26 πŸ”— DoubleJ Nemo_bis: Network functionality isn't free :) If the CPU is spending all its time going "oh shit writewritewrite" it's going to start ignoring some packets.
13:27 πŸ”— Nemo_bis sure, I just don't understand the chronology
13:58 πŸ”— kennethre SketchCow: cut in half
14:04 πŸ”— kennethre 1/3rd
14:05 πŸ”— ersi Nice, kennethre is now #4 on the dashboard
14:05 πŸ”— kennethre I was hoping to be #1 today :)
14:05 πŸ”— tef show off :-)
14:10 πŸ”— kennethre :)
14:11 πŸ”— ersi I'm just sad the dash doesn't show more users :(
16:47 πŸ”— yipdw|m underscor, anyone else who was watching: so how did it take kennethre to get in the top 10 on memac :P
16:47 πŸ”— yipdw|m er, how long
16:48 πŸ”— kennethre yipdw|m: 2 hours :)
16:48 πŸ”— yipdw|m jeez
16:48 πŸ”— kennethre I had to scale back, batcave wasn't doing well
16:48 πŸ”— yipdw|m oh, I see that from the dashboard charts nw
16:48 πŸ”— yipdw|m now
16:49 πŸ”— yipdw|m if you keep the charts at their default scale it's just a vertical line
16:49 πŸ”— kennethre hahaha
16:52 πŸ”— Nemo_bis hehe
16:52 πŸ”— Nemo_bis the graphs should automatically switch to log scale as soon as kennethre appears
16:54 πŸ”— yipdw|m ha
17:41 πŸ”— yipdw man
17:41 πŸ”— yipdw CTA trains today are a mess
17:41 πŸ”— yipdw seems there was a derailment
17:42 πŸ”— yipdw threw off the entire Red Line
20:06 πŸ”— Schbirid oh bollocks, i did not check my robots.txt downloader for weeks and it did not download...
20:07 πŸ”— Schbirid wait, false alarm
20:08 πŸ”— Schbirid nice, 212 days of files
20:21 πŸ”— haesken find anything interesting?
20:22 πŸ”— Schbirid me? i havent poked around in it for months but when i started it was quite fun
20:22 πŸ”— Schbirid i should find a solution for sharing them
20:27 πŸ”— underscor <cryptodragon> "Boob is the only word in English which contains full 3D view of the object. Top view: B. Front View: oo. Side view : b"
20:33 πŸ”— eprillios Hello everyone. :-)
20:34 πŸ”— eprillios I have a question about mobileme-grab.
20:35 πŸ”— eprillios wget-warc has been updated to a new version that allows downloading of bigger files, but should I worry about the data that I've downloaded previously with the old version?
20:36 πŸ”— underscor ask away, eprillios egfault
20:36 πŸ”— underscor whoops
20:36 πŸ”— underscor What would happen before is wget would segfault
20:36 πŸ”— underscor So you'll know if you had any with issues
20:36 πŸ”— underscor They wouldn't get marked as "done" anyway, so it's cool
20:37 πŸ”— eprillios Okay, thank you. :-)
20:37 πŸ”— underscor no :)
20:37 πŸ”— underscor np*
20:37 πŸ”— underscor Thanks for helping out!
irclogger-viewer