#internetarchive.bak 2015-04-10,Fri

↑back Search

Time Nickname Message
00:07 🔗 db48x bah
00:12 🔗 kyan has joined #internetarchive.bak
00:24 🔗 closure ok, so.. only 5 of the repos in shard1 have run iabak lately, it seems. the other 30-some could all be deleted for all I know
00:25 🔗 closure and I'll bet some of them are, there are 7 different repos that all claim to be at root@katie:~/src/IA.BAK/shard1
00:25 🔗 closure http://iabak.archiveteam.org/stats/SHARD1.expire
00:25 🔗 closure if you see yoursef in this list, you're not reporting in and are a candidate for expire
00:28 🔗 SketchCow Nice on the shard progress
00:28 🔗 db48x closure: I meant to ask you about that one; I've run fsck in that repo and synced, but I'm still in there
00:29 🔗 closure which repo is it?
00:29 🔗 db48x db48x@celebdil
00:29 🔗 closure does it have the same annex.uuid as is listed there?
00:30 🔗 SketchCow closure: Sounds like we need to start getting some sort of mailing list/registration going.
00:30 🔗 SketchCow I'd prefer not to start expiring repos just yet.
00:30 🔗 closure SketchCow: agreed, not sure where to store the emails though (not in git I know that)
00:30 🔗 SketchCow I'd like us to go through the logs in this channel, and we can find which names added things.
00:31 🔗 db48x yes
00:32 🔗 closure db48x: hmm. sure you've fscked since upgrading to a new enough git-annex?
00:32 🔗 db48x probably? dunno
00:32 🔗 db48x I'll rerun it
00:33 🔗 SketchCow closure: Mailing list?
00:33 🔗 closure activity.log | 16 +---------------
00:33 🔗 closure woah!
00:34 🔗 closure someone ended up committing a change that nuked 16 fscks out of the activity log
00:34 🔗 closure that's really strange
00:35 🔗 db48x heh
00:35 🔗 closure I'm guessing that means this new feature has a bug..
00:35 🔗 db48x sounds like it
00:42 🔗 * closure reproduced it
00:43 🔗 closure great thing about working with git based stuff, so much info and reproducability..
00:46 🔗 db48x that's a good one to find
00:48 🔗 db48x ooh, interesting
00:48 🔗 db48x http://hastebin.com/raw/zifuhiquja
01:02 🔗 closure ok, that expiry is fixed, but I've already done a release today, so it'll be a while
01:03 🔗 closure maybe I'll switch iabak to the git-annex daily builds for a while
01:11 🔗 closure db48x: well spotted, fixing that too, although it won't cause any problems
01:46 🔗 yipdw closure: weird that 09fde827-7a62-42c7-ab92-f7442e7fb289 ubuntu@iabak:~/IA.BAK/shard1 is on the list
01:46 🔗 yipdw AFAIK that one is still syncing
01:47 🔗 yipdw do I have to do something else to get it to report in?
02:10 🔗 closure yipdw: turns out the code was broken. So, will need to wait until there's a new git-annex to use
02:24 🔗 wp494 has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES)
02:24 🔗 wp494 has joined #internetarchive.bak
03:01 🔗 ppiixx has quit IRC (Remote host closed the connection)
03:02 🔗 ppiixx has joined #internetarchive.bak
03:15 🔗 closure *** Please restart your iabak's ***
03:16 🔗 closure or just run install-git-annex by hand
03:24 🔗 yipdw closure: ah ok
03:33 🔗 niyaje4 has quit IRC (Ping timeout: 600 seconds)
03:55 🔗 niyaje4 has joined #internetarchive.bak
04:36 🔗 chfoo- has joined #internetarchive.bak
04:42 🔗 chfoo- has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in)
04:45 🔗 chfoo- has joined #internetarchive.bak
04:51 🔗 wp494 has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 Start has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 matthusb- has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 Sanqui has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 underscor has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 aschmitz has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 csssuf has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 DFJustin has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 pikhq has quit IRC (ircd.choopa.net ircd.shaw.ca)
04:51 🔗 achip has quit IRC (Read error: Operation timed out)
04:51 🔗 SN4T14__ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 chfoo has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 espes___ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Quile has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 SketchCow has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 closure has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 ersi_ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 bpye_ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 serapeum has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 svchfoo3 has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 destrudo has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Cameron_D has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 ppiixx has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 hatseflat has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 balrog has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Muad-Dib has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 lhobas has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 mrfoo has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 jbenet_ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 ryang has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 antomatic has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Kazzy has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 edsu_ has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 chfoo- has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 hater has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Senji has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 garyrh has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 raylee has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 svchfoo2 has quit IRC (ircd.choopa.net ny.us.hub)
04:51 🔗 Atluxity has quit IRC (ircd.choopa.net ny.us.hub)
04:54 🔗 wp494 has joined #internetarchive.bak
04:54 🔗 Start has joined #internetarchive.bak
04:54 🔗 matthusb- has joined #internetarchive.bak
04:54 🔗 pikhq has joined #internetarchive.bak
04:54 🔗 DFJustin has joined #internetarchive.bak
04:54 🔗 csssuf has joined #internetarchive.bak
04:54 🔗 aschmitz has joined #internetarchive.bak
04:54 🔗 underscor has joined #internetarchive.bak
04:54 🔗 Sanqui has joined #internetarchive.bak
04:54 🔗 ircd.shaw.ca sets mode: +ooo Start pikhq DFJustin
05:15 🔗 niyaje4 has quit IRC (Ping timeout: 601 seconds)
07:10 🔗 zottelbey has joined #internetarchive.bak
07:10 🔗 chfoo- has joined #internetarchive.bak
07:10 🔗 ppiixx has joined #internetarchive.bak
07:10 🔗 raylee has joined #internetarchive.bak
07:10 🔗 SN4T14__ has joined #internetarchive.bak
07:10 🔗 svchfoo2 has joined #internetarchive.bak
07:10 🔗 Senji has joined #internetarchive.bak
07:10 🔗 ersi_ has joined #internetarchive.bak
07:10 🔗 hatseflat has joined #internetarchive.bak
07:10 🔗 balrog has joined #internetarchive.bak
07:10 🔗 Muad-Dib has joined #internetarchive.bak
07:10 🔗 edsu_ has joined #internetarchive.bak
07:10 🔗 jbenet_ has joined #internetarchive.bak
07:10 🔗 mrfoo has joined #internetarchive.bak
07:10 🔗 lhobas has joined #internetarchive.bak
07:10 🔗 bpye_ has joined #internetarchive.bak
07:10 🔗 ryang has joined #internetarchive.bak
07:10 🔗 serapeum has joined #internetarchive.bak
07:10 🔗 Atluxity has joined #internetarchive.bak
07:10 🔗 chfoo has joined #internetarchive.bak
07:10 🔗 antomatic has joined #internetarchive.bak
07:10 🔗 svchfoo3 has joined #internetarchive.bak
07:10 🔗 destrudo has joined #internetarchive.bak
07:10 🔗 Cameron_D has joined #internetarchive.bak
07:10 🔗 espes___ has joined #internetarchive.bak
07:10 🔗 closure has joined #internetarchive.bak
07:10 🔗 ny.us.hub sets mode: +oooo svchfoo2 chfoo svchfoo3 closure
07:10 🔗 SketchCow has joined #internetarchive.bak
07:10 🔗 Quile has joined #internetarchive.bak
07:10 🔗 hater has joined #internetarchive.bak
07:10 🔗 Kazzy has joined #internetarchive.bak
07:10 🔗 ny.us.hub sets mode: +oo SketchCow Kazzy
07:14 🔗 SketchCow http://iabackup.archiveteam.org/ia.bak/ is coming along. I hope we have the space around!
07:15 🔗 SketchCow realeyes: Thanks again for stepping in.
07:17 🔗 garyrh has joined #internetarchive.bak
07:17 🔗 svchfoo1 sets mode: +o garyrh
07:35 🔗 atomotic has joined #internetarchive.bak
08:42 🔗 db48x oops, filled up my root disk :P
09:03 🔗 SketchCow Awwww yis
09:09 🔗 ersi_ is now known as ersi
09:10 🔗 svchfoo2 sets mode: +o ersi
09:30 🔗 ppiixx damn latency, got it up to 1500KB/s with 6 sessions but my line is still mostly idle
09:30 🔗 ppiixx 78GB backed up here so far
09:51 🔗 db48x ppiixx: :)
09:51 🔗 db48x I'm lucky when mine hits 1200
09:51 🔗 ppiixx someone patch in multithreaded wget to git annex ;)
09:54 🔗 db48x closure's working on letting git annex start multiple transfers at once
09:55 🔗 ppiixx cool
09:55 🔗 SketchCow I see an uptick in the graph! Hurrah
09:56 🔗 db48x SketchCow: yea!
10:52 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
11:44 🔗 dirt has quit IRC (Read error: Operation timed out)
11:44 🔗 Lord_Nigh has quit IRC (Read error: Operation timed out)
11:44 🔗 arkiver has quit (Read error: Operation timed out)
11:45 🔗 logchfoo has quit (Read error: Operation timed out)
11:45 🔗 phuzion has quit (Read error: Operation timed out)
11:45 🔗 sep332 has quit (Read error: Operation timed out)
11:45 🔗 dirt (james@[redacted]) has joined #internetarchive.bak
11:46 🔗 Lord_Nigh (~Lord_Nigh@[redacted]) has joined #internetarchive.bak
11:46 🔗 GLaDOS has quit (Read error: Operation timed out)
11:47 🔗 richo has quit (Read error: Operation timed out)
11:47 🔗 S[h]O[r]T has quit (Read error: Operation timed out)
11:47 🔗 marvinw has quit (Ping timeout: 600 seconds)
11:48 🔗 arkiver (~arkiver@[redacted]) has joined #internetarchive.bak
11:48 🔗 svchfoo3 gives channel operator status to arkiver
11:48 🔗 toad1 has quit (Read error: Operation timed out)
11:49 🔗 GLaDOS (~STR_IDENT@[redacted]) has joined #internetarchive.bak
11:49 🔗 svchfoo2 gives channel operator status to GLaDOS
11:49 🔗 Ctrl-S has quit (Read error: Operation timed out)
11:58 🔗 S[h]O[r]T (~Sh]Or]T@[redacted]) has joined #internetarchive.bak
11:58 🔗 phuzion (~phuzion@[redacted]) has joined #internetarchive.bak
11:58 🔗 richo (~richo@[redacted]) has joined #internetarchive.bak
11:59 🔗 marvinw (~marvinw@[redacted]) has joined #internetarchive.bak
11:59 🔗 atomotic (~atomotic@[redacted]) has joined #internetarchive.bak
12:00 🔗 sep332 (~sep332@[redacted]) has joined #internetarchive.bak
12:00 🔗 svchfoo3 gives channel operator status to sep332
12:01 🔗 Ctrl-S (~Ctrl-S@[redacted]) has joined #internetarchive.bak
12:03 🔗 toad1 (~toad@[redacted]) has joined #internetarchive.bak
12:50 🔗 Start has quit (Remote host closed the connection)
12:50 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
12:50 🔗 svchfoo2 gives channel operator status to Start
12:57 🔗 Start has quit (Read error: Connection reset by peer)
12:57 🔗 Start_ (~Start@[redacted]) has joined #internetarchive.bak
12:58 🔗 sankin (~sankin@[redacted]) has joined #internetarchive.bak
13:24 🔗 Start_ has quit (Read error: Connection reset by peer)
13:36 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
13:36 🔗 svchfoo2 gives channel operator status to Start
13:48 🔗 Start has quit (Disconnected.)
13:55 🔗 achip (~thechip@[redacted]) has joined #internetarchive.bak
14:33 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
14:34 🔗 Start has quit (Read error: Connection reset by peer)
14:34 🔗 Start_ (~Start@[redacted]) has joined #internetarchive.bak
14:35 🔗 toad1 (~toad@[redacted]) has left #internetarchive.bak
14:38 🔗 atomotic has quit (Quit: Textual IRC Client: www.textualapp.com)
15:52 🔗 Start_ has quit (Disconnected.)
15:59 🔗 Vito` (sid7616@[redacted]) has joined #internetarchive.bak
16:12 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
16:14 🔗 Start has quit (Read error: Connection reset by peer)
16:14 🔗 Start_ (~Start@[redacted]) has joined #internetarchive.bak
16:53 🔗 Start_ has quit (Ping timeout: 370 seconds)
16:58 🔗 Start-mob (~Start-mob@[redacted]) has joined #internetarchive.bak
17:03 🔗 Start-mob has quit (Remote host closed the connection)
17:47 🔗 atomotic (~atomotic@[redacted]) has joined #internetarchive.bak
17:51 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
17:52 🔗 Start has quit (Read error: Connection reset by peer)
17:52 🔗 Start_ (~Start@[redacted]) has joined #internetarchive.bak
18:27 🔗 atomotic has quit (Quit: My Mac has gone to sleep. ZZZzzz…)
18:33 🔗 Start_ has quit (Disconnected.)
19:02 🔗 Start-mob (~Start-mob@[redacted]) has joined #internetarchive.bak
19:05 🔗 Start-mob has quit (Remote host closed the connection)
19:29 🔗 hater btw i started rewriting IA.BAK in perl
19:31 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
19:32 🔗 Start has quit (Read error: Connection reset by peer)
19:32 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
19:33 🔗 sep332 what's the advantage?
19:37 🔗 hater it'll work on *bsd and osx
19:37 🔗 hater without having to rewrite the bash scripts in a posix-compliant way
19:39 🔗 sep332 we're already having users download git and git-annex. we could add bash too lol
19:41 🔗 hater sep332: no problem: git and git-annex remain the same
19:42 🔗 sep332 i mean have the script download bash binaries and run the rest in bash
19:43 🔗 Vito` osx ships with bash, doesn't putting #!/bin/bash at the top make it execute with bash?
19:43 🔗 hater Vito`: the problem is not bash itself, but the coreutils
19:43 🔗 hater f.e: osx does not support find -printf
19:43 🔗 Vito` ah
19:44 🔗 hater and so on
19:44 🔗 hater there is a huge list of missing features
19:46 🔗 hater some guys on freenode #bash showed me the posix list of minimal subset of supported features and reading the list will take longer than rewriting the script
19:48 🔗 db48x heh
19:49 🔗 SN4T14_ (~SN4T14@[redacted]) has joined #internetarchive.bak
19:51 🔗 SketchCow So, little note.
19:51 🔗 SketchCow Assumptively, X amount of people can code in bash.
19:51 🔗 SketchCow Y people can code in prl.
19:51 🔗 SketchCow Smashing a component of a shared project from one language to another.... that's not a light thing.
19:51 🔗 SketchCow If Y is less than X
19:53 🔗 SN4T14__ has quit (Read error: Operation timed out)
19:53 🔗 hater i understand your point, but atm there are more languages in use: awk, perl, bash and so on
19:58 🔗 sankin has quit (Leaving.)
20:02 🔗 SketchCow In the code we're using?
20:03 🔗 SketchCow Let's put it this way. Put it by closure. If he's all for it, go for it.
20:03 🔗 Start has quit (Disconnected.)
20:03 🔗 SketchCow Otherwise, it should be a IA.BAK-osx script
20:03 🔗 SketchCow closure and db48x are doing the lion's share of hacking this thing
20:07 🔗 hater SketchCow: @closure | well, if you do it in perl, I can help maintain it
20:08 🔗 hater that's what he said yesterday
20:22 🔗 SketchCow OK, concern withdrawn, proceed.
20:23 🔗 Senji coo, what's the huge bulge in 4 copies this afternoon?
20:24 🔗 db48x folks downloading stuff
20:24 🔗 Senji very very quickly
20:25 🔗 Senji There's a couple of nearly vertical lines in that progress graph
20:48 🔗 sep332 how often do clients time out from the graphs?
20:48 🔗 sep332 I haven't been downloading from 03868 (up in New Hampshire) for days but it's still on the map
20:52 🔗 Start-mob (~Start-mob@[redacted]) has joined #internetarchive.bak
20:59 🔗 Start-mob has quit (Remote host closed the connection)
21:20 🔗 Start-mob (~Start-mob@[redacted]) has joined #internetarchive.bak
21:26 🔗 Start-mob has quit (Ping timeout: 370 seconds)
21:31 🔗 hater db48x: http://hastebin.com/gomekohacu.vbs i'm porting iabak-helper and i seriously have no idea what this line is doing?
21:32 🔗 db48x what part of it is unclear?
21:33 🔗 db48x git annex find prints out a bunch of filenames
21:33 🔗 db48x we pipe that to dirname to get the names of the directories that they're in
21:33 🔗 db48x uniq eliminates the duplicates, giving us just the unique directory names
21:34 🔗 db48x shuf shuffles the list
21:34 🔗 db48x git annex get downloads each item in the list
21:34 🔗 hater ok, thx
21:38 🔗 hater is it faster to rewrite some parts of this line in perl or should i just call system("...")?
21:39 🔗 Senji Eww. I'd try and avoid system() as much as possible. Also OSX might not have shuf.
21:39 🔗 hater Senji: i'm seriously NOT going to rewrite git-annex
21:40 🔗 Vito` no shut on yosemite
21:40 🔗 Vito` *shuf
21:40 🔗 hater oh, ok
21:42 🔗 Senji hater: well, no. :-). Executing complex pipelines in system is a recipe for fragility though.
21:42 🔗 pikhq hater: Well, some parts of git-annex are probably quite sensible to reimplement. :)
21:42 🔗 pikhq ... not much used here mind you.
21:44 🔗 yipdw one possibility for shuf on OS X is to also try gshuf; if someone's doing this it's not totally out of the question that they'll also have eg homebrew installed
21:45 🔗 hater yipdw: shuffling a list in perl is no problem
21:45 🔗 yipdw sure
21:46 🔗 yipdw that said the g* pattern has worked before when we didn't want to bother reimplementing coreutils
21:46 🔗 yipdw sure it's not in the base install but meh
21:55 🔗 * closure has concurrent git-annex get working. Not quite ready for general public, but it's gonna be nice!
21:56 🔗 Senji Nomnom
22:06 🔗 Kazzy mmh, dropped 600GB into here overnight, now to see if i can make my home connection pull from IA at anything faster than 300KB/s
22:24 🔗 Start (~Start@[redacted]) has joined #internetarchive.bak
22:24 🔗 svchfoo3 gives channel operator status to Start
22:36 🔗 db48x Kazzy: very nice :)
22:40 🔗 zottelbey has quit (Remote host closed the connection)
23:10 🔗 logchfoo_ starts logging #internetarchive.bak at Fri Apr 10 23:10:10 2015
23:10 🔗 logchfoo_ has joined #internetarchive.bak
23:14 🔗 logchfoo starts logging #internetarchive.bak at Fri Apr 10 23:14:25 2015
23:14 🔗 logchfoo has joined #internetarchive.bak
23:32 🔗 niyaje4 has joined #internetarchive.bak

irclogger-viewer