[02:09] *** DFJustin has quit IRC (Read error: Connection reset by peer) [02:35] *** sevs has quit IRC (Ping timeout: 268 seconds) [03:02] 03registrar 05master fb79b0a 06other 10SHARD12/pubkeys registration of marco_hegenberg on SHARD12 [03:14] *** sep332 has quit IRC (Konversation terminated!) [03:28] *** VADemon has quit IRC (left4dead) [04:16] *** owl has quit IRC (Read error: Operation timed out) [04:46] 03registrar 05master 1a05bec 06other 10SHARD11/pubkeys registration of milenko on SHARD11 [05:04] *** Start has joined #internetarchive.bak [05:38] *** kyan has quit IRC (Quit: Leaving) [06:08] *** owl has joined #internetarchive.bak [07:11] *** milenko has quit IRC (Read error: Connection reset by peer) [07:12] *** milenko has joined #internetarchive.bak [07:59] *** Start has quit IRC (Quit: Disconnected.) [08:19] *** atomotic has joined #internetarchive.bak [08:48] *** db48x has quit IRC (west.us.hub irc.Prison.NET) [08:48] *** coprophag has quit IRC (west.us.hub irc.Prison.NET) [08:48] *** patrickod has quit IRC (west.us.hub irc.Prison.NET) [08:48] *** SketchCow has quit IRC (west.us.hub irc.Prison.NET) [08:48] *** tpw_rules has quit IRC (west.us.hub irc.Prison.NET) [08:49] *** db48x has joined #internetarchive.bak [08:49] *** coprophag has joined #internetarchive.bak [08:49] *** patrickod has joined #internetarchive.bak [08:49] *** SketchCow has joined #internetarchive.bak [08:49] *** tpw_rules has joined #internetarchive.bak [08:49] *** irc.Prison.NET sets mode: +oo SketchCow tpw_rules [08:51] *** pikhq has quit IRC (Read error: Operation timed out) [08:53] *** pikhq has joined #internetarchive.bak [09:55] *** tpw_rules has quit IRC (Read error: Operation timed out) [09:57] *** tpw_rules has joined #internetarchive.bak [09:57] *** svchfoo1 sets mode: +o tpw_rules [10:28] *** SketchCow has quit IRC (Read error: Connection reset by peer) [10:28] *** SketchCow has joined #internetarchive.bak [10:29] *** svchfoo1 sets mode: +o SketchCow [10:36] *** wp494 has quit IRC (Read error: Connection reset by peer) [11:10] 03registrar 05master d5466dd 06other 10SHARD14/pubkeys registration of mail on SHARD14 [11:28] *** wp494 has joined #internetarchive.bak [11:53] 03registrar 05master ceaf633 06other 10SHARD6/pubkeys registration of mail on SHARD6 [11:58] 03registrar 05master a35825f 06other 10SHARD12/pubkeys registration of corin.thorpe on SHARD12 [12:08] *** wp494 has quit IRC (Read error: Operation timed out) [12:08] *** wp494 has joined #internetarchive.bak [12:13] 03registrar 05master 21b1f09 06other 10SHARD6/pubkeys registration of elvian on SHARD6 [13:12] *** AsmoB has joined #internetarchive.bak [13:18] *** AsmoB has quit IRC (Ping timeout: 246 seconds) [13:39] 03registrar 05master af78367 06other 10SHARD14/pubkeys registration of corin.thorpe on SHARD14 [14:11] 03registrar 05master c0046c3 06other 10SHARD18/pubkeys registration of corin.thorpe on SHARD18 [14:58] *** VADemon has joined #internetarchive.bak [15:00] 03registrar 05master 9c5e0e8 06other 10SHARD11/pubkeys registration of corin.thorpe on SHARD11 [15:21] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:25] *** Start has joined #internetarchive.bak [15:42] closure: when does git-annex update it's notion of how many copies a file has? [15:42] db48x: as soon as it verifies it's received (or in some cases) sent a copy of a file [15:43] shardstats is putting that data into graphite every 10 minutes (http://iabak.archiveteam.org:8080/graphlot/?width=869&height=416&_salt=1480606936.833&target=iabak.shardstats.numcopies.0.shard19&lineMode=staircase), but it's almost always the same value every time [15:44] that has to do with propigation of the info from the clients [15:44] only happens when the client pushes git-annex branch to the server, which is iirc only done after it finishes with a shard in iabak [15:45] it doesn't do that on every sync? [15:45] remote: merge git-annex (merging synced/git-annex into git-annex...) [15:45] remote: ok [15:45] To iabak.archiveteam.org:shard2 [15:45] b016505..fa60030 git-annex -> synced/git-annex [15:45] every call to syncshard should do it [15:46] looking at iabak-helper, I'm not sure if syncshard is being called after downloads. Looks like it calls it only before? [15:47] it's called at the end of handleshard [15:47] ah, there's also a background iabak-hourlysync hourly [15:47] right [15:47] so it should be updating pretty regularly [15:47] hourly, not ten-minute-ly [15:48] sure [15:48] I've noticed the hourly task tends to die or get stuck waiting for a lock quite often [15:48] and even when I run the sync manually the next stats run doesn't put in the updated number [15:50] it could be the statss clone of the shard not being updated from the bare repo, or something [15:51] mmm [15:52] yea, that's happenening in /usr/local/IA.BAK/pubkeys/SHARD* [15:53] no, /srv/shard/SHARD* [15:55] no, shardstats-all changes to /usr/local/IA.BAK/pubkeys, then does for SHARD in... [15:56] oh, but then shardstats changes to /srv/shard/whatever; you're right [15:57] it then calls utils/updateclone, which does some stuff [15:58] *** atomotic has joined #internetarchive.bak [16:04] 03registrar 05master e291e5f 06other 10SHARD6/pubkeys registration of mail on SHARD6 [16:06] *** Start has quit IRC (Quit: Disconnected.) [16:11] *** AsmoB has joined #internetarchive.bak [16:13] I guess that part of it looks like it's actually working [16:14] /srv/shard/shard2 shows my last sync in it [16:15] but I did that one manually [16:15] perhaps iabak-hourlysync just rarely works, as Senji says [16:21] perhaps.. you should be able to look at git log git-annex and see the info in there from your client [16:21] if you didn't already [16:28] d8a9713 db48x@iabak Wed Nov 30 10:08:07 2016 -0500 [16:28] 3308a22 db48x@iabak Wed Nov 30 06:23:58 2016 -0500 [16:28] 6151f6c db48x@iabak Wed Nov 30 06:21:47 2016 -0500 [16:28] super hourly [16:42] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [17:41] *** beardicus has joined #internetarchive.bak [17:59] *** db48x has quit IRC (Remote host closed the connection) [18:01] *** db48x has joined #internetarchive.bak [19:04] *** owl has quit IRC (Read error: Operation timed out) [19:06] *** db48x has quit IRC (hub.efnet.us irc.Prison.NET) [19:06] *** tpw_rules has quit IRC (hub.efnet.us irc.Prison.NET) [19:06] *** coprophag has quit IRC (hub.efnet.us irc.Prison.NET) [19:06] *** patrickod has quit IRC (hub.efnet.us irc.Prison.NET) [19:09] *** milenko has quit IRC (Remote host closed the connection) [19:10] *** milenko has joined #internetarchive.bak [19:10] *** copropha1 has joined #internetarchive.bak [19:35] *** milenko has quit IRC (Read error: Operation timed out) [19:35] *** milenko has joined #internetarchive.bak [19:57] *** db48x has joined #internetarchive.bak [19:57] *** tpw_rules has joined #internetarchive.bak [19:57] *** patrickod has joined #internetarchive.bak [19:57] *** irc.Prison.NET sets mode: +o tpw_rules [20:24] *** milenko has quit IRC (Read error: Operation timed out) [20:25] *** milenko has joined #internetarchive.bak [20:32] *** db48x has quit IRC (Read error: Operation timed out) [20:40] *** db48x has joined #internetarchive.bak [20:56] *** atomotic has joined #internetarchive.bak [21:14] *** AsmoB has quit IRC ((null)) [21:48] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [22:55] *** db48x has quit IRC (Ping timeout: 255 seconds) [23:39] *** balrog has quit IRC (Read error: Operation timed out) [23:39] *** hawc145 has joined #internetarchive.bak [23:42] *** balrog has joined #internetarchive.bak [23:42] *** svchfoo3 sets mode: +o balrog [23:43] *** yipdw has quit IRC (Read error: Connection reset by peer) [23:43] *** thelsdj has quit IRC (Ping timeout: 370 seconds) [23:43] *** HCross has quit IRC (Ping timeout: 370 seconds) [23:43] *** yipdw has joined #internetarchive.bak [23:43] *** svchfoo1 sets mode: +o yipdw [23:43] *** Cameron_D has quit IRC (Ping timeout: 370 seconds) [23:43] *** MrRadar has quit IRC (Ping timeout: 370 seconds) [23:46] *** thelsdj has joined #internetarchive.bak [23:51] *** kurt has quit IRC (Ping timeout: 370 seconds) [23:53] *** kurt has joined #internetarchive.bak [23:57] *** yipdw_ has joined #internetarchive.bak