[00:00] *** balrog has joined #internetarchive.bak [01:28] *** beardicus has joined #internetarchive.bak [01:57] *** beardicus has quit IRC (Quit: Sleep.) [04:30] *** svchfoo2 has quit IRC (Read error: Connection reset by peer) [04:33] *** svchfoo2 has joined #internetarchive.bak [04:40] *** raylee has quit IRC (Ping timeout: 240 seconds) [04:45] *** svchfoo2 has quit IRC (Ping timeout: 240 seconds) [04:47] *** Atluxity has quit IRC (Ping timeout: 360 seconds) [04:47] *** Atluxity has joined #internetarchive.bak [04:49] *** svchfoo2 has joined #internetarchive.bak [04:49] *** svchfoo1 sets mode: +o svchfoo2 [04:51] *** raylee has joined #internetarchive.bak [05:27] *** wp494 has quit IRC (Ping timeout: 740 seconds) [05:32] *** wp494 has joined #internetarchive.bak [05:35] *** zottelbey has joined #internetarchive.bak [06:59] *** niyaje4 has joined #internetarchive.bak [07:35] *** atomotic has joined #internetarchive.bak [07:46] *** wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) [07:56] *** niyaje4 has quit IRC (Ping timeout: 600 seconds) [08:30] *** niyaje4 has joined #internetarchive.bak [09:28] *** niyaje4 has quit IRC (Ping timeout: 600 seconds) [09:40] *** wp494 has joined #internetarchive.bak [10:18] *** beardicus has joined #internetarchive.bak [11:12] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [12:00] *** atomotic has joined #internetarchive.bak [13:02] *** Start has quit IRC (Disconnected.) [13:04] *** Start has joined #internetarchive.bak [13:04] *** svchfoo1 sets mode: +o Start [13:23] *** sankin has joined #internetarchive.bak [13:43] *** Start has quit IRC (Read error: Connection reset by peer) [14:59] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:11] *** Start has joined #internetarchive.bak [15:19] *** toad2 has joined #internetarchive.bak [15:27] *** toad1 has quit IRC (Read error: Operation timed out) [15:51] *** Start has quit IRC (Disconnected.) [15:59] *** Start has joined #internetarchive.bak [16:02] *** berndj has quit IRC (Read error: Operation timed out) [16:40] *** Start has quit IRC (Disconnected.) [16:55] *** Start has joined #internetarchive.bak [17:06] *** Start has quit IRC (Disconnected.) [17:47] *** garyrh has quit IRC (hub.se irc.ac.za) [18:43] *** Start has joined #internetarchive.bak [18:45] Hi, it's time for jason questions! [18:46] 1. Have we begun hurting Shard1 to see responsiveness by the clients? [18:46] 2. Should we be adding more shards, for people offering more than 6tb of space? [18:46] 3. Should we be forcing central registration now? [18:59] *** garyrh has joined #internetarchive.bak [18:59] *** svchfoo1 sets mode: +o garyrh [19:01] 2, yes I was thinking the same [19:01] 3, yes but someone has to write it [19:01] 1, still waiting for some clients to upgrade from a buggy version that is preventing this from working [19:05] Good on all. [19:05] I gave you some collections to add. need them again? [19:07] I didn't see that [19:07] *** Start has quit IRC (Disconnected.) [19:13] btw, I don't know if you saw this guy http://iabak.archiveteam.org/stats/ALL.leaderboard [19:23] I'm surprised how high up I am on there. [19:25] SketchCow: what were the collections? [19:26] I saw that! Obviously it helps me when the stuff is graphical, but it's great to see it. [19:27] graphical version needs work, but it shows that only 2 people are working on SHARD2 right now [19:27] http://iabak.archiveteam.org:8080/render/?width=1060&height=717&_salt=1428872058.728&target=keepLastValue%28highestAverage%28iabak.shardstats.leaderboard.*.shard2%2C10%29%29&from=-24hours [19:28] *** Start has joined #internetarchive.bak [19:33] *** atomotic has joined #internetarchive.bak [19:33] i'm pulling slowly from IA on shard2.. maybe 300KB/s [19:33] yeah, I see a few slow pulls on closer inspection [19:33] I'm e7971 [19:36] probably won't be going up any more though [19:36] any idea who e3faaa97 is? [19:39] * pikhq is ca3d2663; wheee. [19:40] Kazzy: oh, that's on shard1; looks like one of closure's [19:40] ah right, just seemed weird as it has no user/host etc on the leaderboard [19:40] Kazzy: you can run "git annex info" to see information about what other users there are [19:41] that is an interesting case.. that one does not show up in uuid.log [19:41] so I don't know who it is, or how they got initialized w/o that [19:41] oh, I misread the number [19:41] yea, that is odd [19:41] they are, however, sending activity pings [19:42] so still out there [19:42] (this is why we need registration) [19:42] maybe that info got removed? [19:43] no, I checked all versions [19:43] aha, it's midas [19:43] * closure found it in another log ;) [19:44] midas: curious how you managed that trick :) [19:44] closure: one of mine failed with this error: https://pastebin.mozilla.org/8829881 [19:45] btw, you can run this to set your repo's description to something else: git annex describe . my name here [19:45] db48x: that's an overlap with someone else pushing at the same time [19:45] I thought it might be [19:45] perhaps it should retry? [19:45] it'll automatically clear up [19:45] I have a more robust method that the assistant uses, and I'm considering enabling it here too [19:47] *** Start has quit IRC (Disconnected.) [19:48] did what closure ? [19:48] I guess we can keep an eye on it [19:48] I wonder why mine has such a large jump in it [19:49] did I download one file that was 250+ GB? [19:49] midas: somehow ended up with your repo not having a known description [19:49] or did I have a long string of failures to commit? [19:49] *** SN4T14__ has joined #internetarchive.bak [19:49] lol [19:51] db48x: or, maybe it sped up a whole lot one hour [19:51] found out why i'm not on the map: freegeoip has got nothin' for my IP. [19:52] sends back coordinates for kentucky i think (wrong), and no postcode. [19:52] oh, no... kansas city. [19:53] *** SN4T14_ has quit IRC (Ping timeout: 306 seconds) [19:55] closure: I wish! [19:57] *** zottelbey has quit IRC (Remote host closed the connection) [20:03] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [20:03] http://iabak.archiveteam.org/stats/SHARD1.expireleaderboard [20:04] http://iabak.archiveteam.org:8080/render/?width=1060&height=717&_salt=1428955433.602&target=secondYAxis%28keepLastValue%28iabak.shardstats.filestransfered.shard2%29%29&target=highestAverage%28keepLastValue%28iabak.shardstats.leaderboard.*.shard2%29%2C10%29&hideLegend=false [20:04] so, that's looking better.. we see some repos that were never used and can be expired, and some repos that we need to get in touch with the owners, who are apparently not running iabak [20:05] closure: we need to make it easier to keep syncing regularly even when not running iabak itself [20:05] yes, need a cron job or something [20:05] what is the second Y axis there? [20:05] oh, I see [20:06] it wasn't just 1 file then was it [20:06] nope [20:06] *** atomotic has joined #internetarchive.bak [20:10] closure - Let me give you some, and make enough for it to be a sizeable shard. [20:10] you might go for 2 shards [20:11] https://archive.org/details/usfederalcourts [20:13] oh, that's the 13 million file one. 130 shards.. [20:14] Oh, is THAT what held you back. [20:14] OK, let's hold off on that. [20:17] https://archive.org/details/78rpm [20:17] https://archive.org/details/gutenberg [20:19] 78rpm .. nice, 100k files [20:20] heh [20:21] gutenburg is 600k [20:24] :) [20:26] so, let's see how big a shard of just records would be [20:34] . o O ( another shard my inner packrat wants a copy of all of ) [20:36] gutenberg is mirrored lots of other places [20:45] 1.47 terabytes of 78's!!! [20:45] and, that actually came out to 95k files, so we could add another 5-15k files [20:46] oldtimeradio? [20:46] wait, I'm reading the wrong thing [20:53] closure: re: buggy clients -- what version should we be running right now? [20:53] I'm on 5.20150409-gd3d92ab, just want to confirm that that's fine [20:53] also I guess I can shove the right version in the topic or something [20:54] *** sankin has quit IRC (Leaving.) [21:03] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [21:30] *** berndj has joined #internetarchive.bak [21:46] *** niyaje4 has joined #internetarchive.bak [22:08] ok, real number for 78rpm: 240gb [22:08] which is fine, if we can find some big files to round it out [22:47] oldtimeradio is a good one too [22:47] It's one of our most popular sets [22:48] *** Start has joined #internetarchive.bak [22:49] *** svchfoo3 sets mode: +o Start