#archiveteam 2014-09-07,Sun

↑back Search

Time Nickname Message
00:22 🔗 SketchCow It will not be a petabyte.
04:55 🔗 chfoo emergency tracker maintaince in progress
04:55 🔗 xmc what's the situation?
04:55 🔗 chfoo redis oom'ed
04:55 🔗 xmc if you reboot it, we can maybe upgrade to more ram
04:56 🔗 xmc let me know and I'll poke the upgrade button on linode's page thing
04:56 🔗 chfoo i thought swap was supposed to prevent this
04:56 🔗 xmc that is, if you're going to upgrade it
04:56 🔗 xmc hahahaha swap
04:56 🔗 xmc oh
04:56 🔗 xmc Linodes with configuration profiles referencing a 32 bit kernel are currently not eligible for this upgrade. For more information please see our switching kernels guide, or redeploy this Linode using a 64 bit distro.
04:57 🔗 xmc we should schedule a few hours tracker downtime
04:57 🔗 Rotab its down anyways so... :P
04:57 🔗 xmc I did it on another machine and it was fairly straightforward
04:57 🔗 xmc would rather do this kind of thing while not under duress
04:58 🔗 xmc chfoo: do you have access to the linode dashboard that the tracker is in?
04:58 🔗 chfoo i don't think so. do i need it?
04:58 🔗 yipdw actually, if you're going to do that
04:58 🔗 yipdw er wait a minute nm
04:59 🔗 yipdw I was going to suggest merging my weighted upload tracker PR in and deploying it, but the data structure is incompatible with existing trackers
04:59 🔗 yipdw chfoo: are you draining done sets?
04:59 🔗 yipdw oh yes
04:59 🔗 yipdw I see
05:00 🔗 yipdw jesus, why is that so huge
05:00 🔗 chfoo i just drained verizon
05:00 🔗 yipdw (integer) 2878339
05:00 🔗 chfoo i think the claims for it are just too big
05:00 🔗 yipdw never mind, answered my own question
05:00 🔗 yipdw redis 127.0.0.1:6379> zcard verizon:out
05:00 🔗 yipdw yipdw@shilling:~$ redis-cli
05:01 🔗 chfoo yeah
05:01 🔗 yipdw our cruisers cannot repel firepower of that magnitude
05:01 🔗 xmc we're gonna need a bigger box
05:01 🔗 yipdw bigger problem is that you can't load the claims page for verizon anymore
05:01 🔗 chfoo huh, node.js got killed too
05:01 🔗 yipdw even if the tracker were online
05:02 🔗 xmc speaking of scheduling downtime, I might want to move the tracker to a different vps hosting co.
05:02 🔗 xmc way down the priority list
05:03 🔗 yipdw haha shit
05:03 🔗 yipdw I forgot, this is Redis 2.6
05:03 🔗 yipdw (error) ERR unknown command 'hscan'
05:03 🔗 yipdw hscan verizon:claims
05:03 🔗 yipdw so even manually releasing those claims is going to be a problem
05:03 🔗 yipdw hmmmmmm
05:04 🔗 chfoo the log drainer and updates channel are up again. i'm done what i was going to do other than bring back up nginx
05:04 🔗 yipdw I'm going to try and reduce the out/claims sets
05:05 🔗 yipdw shutting down redis, one moment
05:05 🔗 yipdw quick fix for memory fragmentation
05:06 🔗 xmc brute force defragmentation
05:06 🔗 yipdw is there any other way to do it?
05:06 🔗 * xmc shrugs
05:06 🔗 yipdw i don't know of any malloc that is smart enough to defragment its heaps
05:06 🔗 yipdw well, jemalloc does, rarely
05:07 🔗 yipdw I guess copying GCs inherently defragment but whether or not they return the free memory to the OS is a different question
05:07 🔗 xmc ah yeah
05:10 🔗 yipdw oh yeah
05:10 🔗 yipdw if we are going to schedule downtime, I want to get the tracker support scripts running in daemontools or runit
05:10 🔗 yipdw running a tracker without the log drainer is bad news
05:11 🔗 * xmc nods
05:11 🔗 * crypto__ pretends to undertstand all these words
05:12 🔗 xmc scheduling downtime is hard to do when projects are so unpredictable :P
05:12 🔗 yipdw maybe we need tracker1 and tracker2
05:12 🔗 xmc now there's a good idea
05:12 🔗 chfoo does redis support distributed?
05:12 🔗 xmc why don't we spin up a new tracker and administer it properly, then shift load over
05:13 🔗 yipdw chfoo: uh kinda
05:13 🔗 yipdw multiple read is fine
05:13 🔗 xmc :|
05:13 🔗 yipdw but for multiple masters you really need redis-cluster or consistent hashing for writes etc
05:13 🔗 yipdw the latter is probably easier for us to do
05:13 🔗 yipdw especially since the Ruby Redis client already supports it
05:14 🔗 yipdw consistent hashing for sharding that is
05:14 🔗 yipdw it'll require changes to the tracker, but I'm already working in there anyway
05:14 🔗 yipdw tracker could really use a good dose of redis-namespace
05:15 🔗 xmc when would you guys like another machine by?
05:15 🔗 yipdw -1 hour
05:15 🔗 xmc how about tomorrow
05:15 🔗 yipdw no seriously any time is fine
05:15 🔗 xmc :P
05:15 🔗 chfoo i still have the puush tracker running if it's needed
05:15 🔗 xmc I recently joined a xen collective, so I can provision machines more cheaply
05:15 🔗 yipdw how much power does that have?
05:18 🔗 yipdw shit, I wonder if I can even HKEYS the verizon claims hash
05:18 🔗 yipdw you know, at this point, I think I should just delete verizon:claims and just add everything back to verizon:todo
05:18 🔗 yipdw comments?
05:19 🔗 chfoo would it be possible to create a new project under the tracker and shove the claims to that one?
05:19 🔗 yipdw yeah, but I'd still be unable to read the claims
05:19 🔗 yipdw there's just too much dataw
05:20 🔗 yipdw read them via the tracker UI that is
05:20 🔗 yipdw I do have all the keys in the hash, so I can start trimming them that wa
05:20 🔗 yipdw y
05:20 🔗 chfoo oh, well, i don't really know. you should ask Arkiver2
05:20 🔗 yipdw ok
05:20 🔗 yipdw Arkiver2: ^
05:21 🔗 yipdw if we told antirez what we're doing with redis he'd probably say that we're fucking idiots
05:21 🔗 yipdw heh
05:22 🔗 xmc lol
05:23 🔗 chfoo if anyone doesn't know, he personally contacted me to fix a data losing bug when i got mad at redis
05:25 🔗 yipdw oh?
05:28 🔗 chfoo i didn't read the faq about the overcommit setting and for days redis didn't do a dump of the db and then it oom'ed.
05:30 🔗 yipdw ahh
05:33 🔗 chfoo http://osdir.com/ml/Redis/2013-08/msg00292.html
05:33 🔗 chfoo anyway, i won't be touching the tracker. i have to sleep now
05:35 🔗 yipdw 'night
05:36 🔗 yipdw hm
05:37 🔗 yipdw ok
05:42 🔗 yipdw so I just released all verizon claims <= 2014-09-04 00:00:00Z
05:42 🔗 yipdw hopefully this makes things not suck
07:47 🔗 SketchCow Boop
07:49 🔗 BlueMaxim Bop
07:49 🔗 xmc brap
14:34 🔗 Kazzy any update on tracker status?
16:58 🔗 Arkiver2 http://assets.www.npo.nl/uploads/media_item/media_item/62/66/Digitaal_geheugen_verlies_3_cover-1409907995.jpg
16:58 🔗 Arkiver2 This evening on NPO2 in the Netherlands at 21:05 (Dutch time) SketchCow will be online in a documentary
16:59 🔗 Arkiver2 see the preview here: http://www.npo.nl/vpro-tegenlicht/07-09-2014/VPWON_1209790
17:12 🔗 schbirid is there a well-working way to archive single pieces content from facebook (statuses or images with comments)?
17:15 🔗 godane Arkiver2: is that a sombrero hat in SketchCow room?
17:16 🔗 Arkiver2 godane: hah, yep!
17:59 🔗 Arkiver2 midas joepie91: don't forget to watch SketchCow this evening! NPO2 21:05
18:19 🔗 SketchCow I just watched it.
18:19 🔗 SketchCow (They sent me a vimeo link)
18:19 🔗 SketchCow It's good. I wish they hadn't shown the front of my house.
18:20 🔗 Arkiver2 SketchCow: I still have to watch it, excited :)
18:22 🔗 xmc typical euros, care about their own privacy but not others'
18:26 🔗 godane SketchCow: i figured a way to archive videos from npo.nl
18:26 🔗 godane it maybe use full for your video if anything else
18:36 🔗 godane is there a english version of npo.nl site
18:36 🔗 godane without google translater
18:36 🔗 godane i only ask cause they are put up videos with english audio
18:47 🔗 pluesch someone with admin rights on the wiki here? :/
18:49 🔗 pluesch the problem is: I can't change my password, because I don't know it anymore. I still have a the session cookies to "authorize" but they will expire soon.
18:54 🔗 Arkiver2 godane: I'll try to get subtitles
19:10 🔗 godane looks like in mplayer ts from npo moves too fast
19:10 🔗 godane xine plays them fine
19:18 🔗 pluesch godane: version?
19:23 🔗 godane ?
19:23 🔗 godane mplayer is 1.1
19:24 🔗 antomatic I swung my satellite dish around but sadly NPO2 is encrypted. Buh.
19:26 🔗 pluesch maybe try mplayer2
19:33 🔗 Rotab they blocked DO AMS ips :(
19:33 🔗 Tikatu A question for y'all.
19:34 🔗 Tikatu A mirror site for fanfiction.net has been noticed. Do y'all have anything to do with it?
19:38 🔗 xmc Tikatu: link?
19:38 🔗 Tikatu http://www.bluebuffalodog.com/
19:39 🔗 xmc nah
19:39 🔗 xmc if it's archiveteam, it'd say archiveteam
19:39 🔗 Tikatu Didn't think so but wanted to be sure. Thanks heaps!
19:39 🔗 Tikatu Now I'm off to notify Xing Li that he's been pirated.
19:40 🔗 xmc who?
19:40 🔗 Tikatu Again.
19:40 🔗 xmc oh
19:40 🔗 xmc ff.n guy
19:40 🔗 schbirid pluesch: edit the cookies expiration date: )
19:40 🔗 xmc he doesn't own that shit
19:40 🔗 Tikatu The owner of fanfic.net
19:40 🔗 xmc he owns the brand and the site
19:40 🔗 xmc not the stories :]
19:40 🔗 Tikatu Exactly
19:40 🔗 Tikatu I'
19:40 🔗 xmc and it doesn't look like any of that shit got pulled over
19:41 🔗 xmc *shrug*
19:41 🔗 DFJustin here at archive team we believe the more mirrors the better
19:41 🔗 xmc ^
19:41 🔗 pluesch schbirid: that will work?
19:41 🔗 Tikatu I'm glad y'all archived as much as you did; there's a lot of plagiarists who are being axed because they're being reported to the authors' publicists.
19:41 🔗 schbirid pluesch: i bet it does
19:42 🔗 Tikatu If JK Rowling suddenly said she wasn't okay with fanfiction anymore, there goes half the site.
19:42 🔗 pluesch schbirid: okay. thank you :3
19:44 🔗 Tikatu Later! :)
20:10 🔗 pluesch meh ... tumblr api TOS forbids archiving -.-
20:12 🔗 godane SketchCow: you in high res: http://images.poms.omroep.nl/image/s2000/506379.png
20:12 🔗 godane so maybe able to archive a ton of high res screenshots that way of npo
20:15 🔗 bsmith093 godane, any project currently running?
20:20 🔗 godane got to channel #quitpic
20:21 🔗 brooky12 what is this project for?
20:21 🔗 godane we are trying to save twipic
20:21 🔗 godane *twitpic
20:21 🔗 brooky12 ah
20:21 🔗 godane http://archiveteam.org/index.php?title=TwitPic
20:26 🔗 bzc6p_ Don't forget #verizoff, Verizon Personal Web space archiving project
20:32 🔗 godane SketchCow: that no downloading if not 'login bug' is stage6 now
20:32 🔗 godane don't know if that intended or a bug
21:24 🔗 antomatic That documentary is great
23:32 🔗 ronalde WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD
23:35 🔗 BlueMaxim yahoosucks
23:36 🔗 ronalde BlueMaxim: thanks (or art thou a bot?)
23:36 🔗 BlueMaxim not a bot, just one of the people who thinks yahoo sucks :P

irclogger-viewer