[01:16] *** Beardface has quit IRC (Remote host closed the connection) [01:44] 03registrar 05master e647d92 06other 10SHARD7/pubkeys registration of iabak on SHARD7 [01:47] So what happens to the expired mess on shard3? [01:52] *** Beardface has joined #internetarchive.bak [02:25] 03registrar 05master bb9d306 06other 10SHARD7/pubkeys registration of rtucker-iabak on SHARD7 [02:57] 03registrar 05master b065047 06other 10SHARD8/pubkeys registration of tomhiggins on SHARD8 [03:21] *** garyrh has quit IRC (Read error: Operation timed out) [03:47] *** wp494 has quit IRC (Remote host closed the connection) [03:48] *** wp494 has joined #internetarchive.bak [04:19] *** garyrh has joined #internetarchive.bak [06:04] *** zottelbey has joined #internetarchive.bak [06:42] *** zz_CyberJ is now known as CyberJaco [07:30] *** CyberJaco is now known as zz_CyberJ [07:35] *** primus104 has joined #internetarchive.bak [07:53] *** hendi_ has joined #internetarchive.bak [08:36] *** primus104 has quit IRC (Leaving.) [08:38] *** primus104 has joined #internetarchive.bak [08:45] tpw_rules: your shard8 isn't registered [09:13] * db48x sighs [09:13] I just typed "got ammex dorp" [09:14] That's how Harry Potter defeated Voldemort [09:17] heh [09:18] OK, bed [09:18] Man, what a mess of a trip home [09:20] take the train next time; longer but more relaxing [10:17] 03registrar 05master 5a79c11 06other 10SHARD7/pubkeys registration of fruitbaticus on SHARD7 [10:45] *** primus104 has quit IRC (Leaving.) [11:42] *** hendi_ has quit IRC (west.us.hub irc.eversible.com) [11:42] *** zottelbey has quit IRC (west.us.hub irc.eversible.com) [11:42] *** bpye has quit IRC (west.us.hub irc.eversible.com) [11:42] *** svchfoo3 has quit IRC (west.us.hub irc.eversible.com) [12:08] *** hendi_ has joined #internetarchive.bak [12:08] *** zottelbey has joined #internetarchive.bak [12:08] *** bpye has joined #internetarchive.bak [12:13] *** mariusz has joined #internetarchive.bak [12:34] *** hendi__ has joined #internetarchive.bak [12:40] *** hendi_ has quit IRC (Ping timeout: 512 seconds) [12:46] *** sankin has joined #internetarchive.bak [13:13] *** phuzion has quit IRC (Remote host closed the connection) [13:13] *** phuzion has joined #internetarchive.bak [13:29] *** primus104 has joined #internetarchive.bak [13:37] *** zottelbey has quit IRC (Quit: Leaving) [13:53] *** Start has quit IRC (Disconnected.) [14:00] 03registrar 05master c6ea8db 06other 10SHARD3/pubkeys registration of jdamery+iabak on SHARD3 [14:12] *** svchfoo3 has joined #internetarchive.bak [14:13] *** svchfoo2 sets mode: +o svchfoo3 [14:29] *** primus104 has quit IRC (Leaving.) [14:32] *** Start has joined #internetarchive.bak [14:54] *** Start has quit IRC (Disconnected.) [15:01] http://iabak.archiveteam.org/SHARD3.html [15:01] So, I see the shard lost the supporting item, but there's been no effort by The System to heal it, even as people are filling in later shards. [15:03] *** Start has joined #internetarchive.bak [15:03] The shard is still on 'maint' status, and it's been long enough probably noone's doing downloads (I've just set up a new checkout of the shard, but that won't download until tonight) [15:03] I don't think there's an automated process for going from 'maint' to 'active' yet [15:04] Yeah, it seems like one should run every 24 hours or something. [15:05] I think this is the first time we've had a big expire though [15:05] One this large, yes. [15:06] But that's why we're doing things as we're doing them, to learn all the situations and make code or bugfixes to handle them. [15:06] I've reached out to The Public but this thing still requires reasonable knowledge. [15:06] *nod* [15:06] When we move to "you have a desktop and a dock and a pile of hard drives" it will be different. [15:06] Looks as if there'll be another big expire on shard3 shortly too [15:06] Then I'll join too! [15:07] Sadly for me, all my hosted disk space is the Internet Archive itself. [15:07] * SketchCow cries eagle tear [15:08] heh [15:08] *** sep332 has quit IRC (Read error: Operation timed out) [15:09] I started a patch for that last night [15:10] but I stopped when I noticed that it shouldn't be necessary [15:11] if you have the cronjob/systemd service running then after it fscks the shard it will check to see if there are any files that need to be downloaded [15:11] I thought the cronjob wasn't supposed to download files? [15:11] no, cron job does not download [15:12] it's there on line 151 :P [15:13] if [ ! "$CRONJOB" ]; then [15:13] oh, it's inverted [15:15] *** sep332 has joined #internetarchive.bak [15:15] *** svchfoo1 sets mode: +o sep332 [15:15] we should remove that check [15:15] well, no. the cron job is supposed to run fast and not use up more disk [15:15] Don't feed cronjob after midnight [15:16] then we'll never recover from expirations unless the users manually rerun iabak [15:17] better to set the shard back to active [15:17] that won't change anything [15:17] shard has lost one contributor; it should gain one or more new contributors to make up for it [15:17] oh, I guess new users could come along [15:18] asking the existing contributors to shoulder the load will be increasinly onerous [15:18] did I just use that word on irc? [15:19] * closure made up for it by hick-spelling increasin'ly [15:19] :) [15:19] onerous is a good word [15:21] *** Start has quit IRC (Disconnected.) [15:27] These are, to again say it, exactly the type of situations I want us to deal with and understand. [15:28] I realize we may get to a point that some place offers 300tb [15:39] yeah, i would gladly start downloading shard3, but i can't get one of the machines to stop downloading shard6 [15:39] :/ [15:44] Downloading is what's important - let us figure out how to handle the situation cleanly and clearly. [15:49] yeah, but I don't want to download shard6 since that's already in maint [15:49] :) [15:49] well, at least the other machine downloads shard8 so that's ok [15:51] you can just hit control-c whenever you want [15:52] back later [16:02] *** Start has joined #internetarchive.bak [16:11] i many times:) [16:33] Senji: really? blech [16:33] is there a way to register just that one [16:35] 03registrar 05master 4760bf4 06other 10SHARD1/pubkeys registration of twatson52 on SHARD1 [16:35] 03registrar 05master aa0b983 06other 10SHARD5/pubkeys registration of twatson52 on SHARD5 [16:35] perhaps make it so that registration doesn't cause a new commit for the same uuid, email, pubkey pair? [16:35] 03registrar 05master be8191b 06other 10SHARD6/pubkeys registration of twatson52 on SHARD6 [16:36] 03registrar 05master 272e175 06other 10SHARD8/pubkeys registration of twatson52 on SHARD8 [16:44] *** Start has quit IRC (Disconnected.) [16:49] *** primus104 has joined #internetarchive.bak [17:40] As of this current moment, The Internet Archive has 18,198,300 torrents available with 34,912 peers contributing bandwidth. [17:41] great stat in light of the suggestion that bittorrent be used for ia.bak .. [17:42] seems like it would not help our downloads of much ... [17:43] otoh, seeding from iabak repos is doable [17:43] Well, when you use a bittorrent, you use both our datacenters. [17:43] closure: It's a thought to make that an option. Opt-in, please. [17:43] OPT. IN [17:43] But yes, "would you like to contribute bandwidth" [17:47] *** sep332 has quit IRC (Read error: Operation timed out) [17:48] *** Start has joined #internetarchive.bak [17:49] *** sep332 has joined #internetarchive.bak [17:49] *** svchfoo3 sets mode: +o sep332 [17:51] *** hendi__ has quit IRC (Ping timeout: 512 seconds) [17:56] *** sankin has quit IRC (Leaving.) [18:10] *** sankin has joined #internetarchive.bak [18:35] *** zz_CyberJ is now known as CyberJaco [18:36] *** Start has quit IRC (Disconnected.) [19:09] *** VADemon has joined #internetarchive.bak [19:55] *** espes__ has quit IRC (Ping timeout: 186 seconds) [20:06] *** espes__ has joined #internetarchive.bak [20:19] *** mariusz has quit IRC (Read error: Operation timed out) [20:51] *** sankin has quit IRC (Leaving.) [20:57] *** mehblahme has joined #internetarchive.bak [21:00] *** mehblahme has quit IRC (Client Quit) [23:03] *** CyberJaco is now known as zz_CyberJ [23:18] *** Start has joined #internetarchive.bak [23:45] are shards going to become re-active as people expire? [23:49] also annex.sshcaching = false should carry between shards [23:59] how is shard8 already having clients expire?