[01:05] uhh [01:05] I haven't done shit [01:05] somebody is maybe giving me credit for good work? [01:06] *** mntasauri has quit IRC (Max SendQ exceeded) [01:06] but at some point I do want to help the effort [01:06] I think it's a good cause [01:06] *** mntasauri has joined #internetarchive.bak [01:09] ryang: this seems to be yours: 2207633498105: 8e7f6693-32d1-45f3-9995-b9596aa783de -- ubuntu@ia:/mnt/ia/IA.BAK/shard6 [01:10] closure: can't be. I don't even have 2TB of storage in my house right now [01:10] Unless my buddy hooked up my old NAS, deleted my data, and started it for me [01:10] ryang: atwgpc? [01:11] I don't even know what you mean by that. [01:11] oic. than ryan.g != ryang [01:11] must not be [01:14] btw, ryang.g has pulled another 250 gb since I mentioned him a few hours ago [01:14] my hero. :P [01:14] I wish I could help out [01:14] But my shit has been pretty ruined ever since the fbi ruined my shit. [01:15] ??? [01:15] There's a story in there. [01:19] *** wp494_ is now known as wp494 [01:56] hey, someone want to write something that can, given an IA collection name, return some numeric ranking for the collection? [01:57] number of downloads, or something [01:57] some shards will have a lot of small collections, and it would be nice to sort by popularity for display [02:06] !ao https://ghostbin.com/paste/9hr4r [02:17] *** primus104 has quit IRC (Leaving.) [02:33] *** antomatic has joined #internetarchive.bak [02:37] *** antomati_ has quit IRC (Ping timeout: 370 seconds) [02:50] found a collection that breaks the shard creator. [02:50] amusingly, it's from the CCC [02:50] * closure fixes [02:56] *** VADemon_ has quit IRC (Quit: left4dead) [02:57] okay we are back on line [02:57] sorry for all the registration spam [02:58] closure: git annex seems to like leaving around ssh processes after being ctrl+Ced [02:59] maybe it's iabak's fault, but it doesn't start ssh directly [02:59] there's a background ssh that's used for connection caching [02:59] git config annex.sshcaching false will disable that [02:59] i have that disabled. [03:11] I guess it's just the background iabak-hourlysync [03:19] 8.8 tb shard. so that just happened [03:43] closure: oh dear. i guess it's time to add new disks [03:49] look on the bright side, we're getting close to 1 million files, which is 1/271th of all the files [03:50] * closure notices that the iabak server only has 15 million free inodes.. will need to do something about that one of these months [03:56] store each shard as a disk image? [03:56] and loop mount it [03:57] *** chazchaz has quit IRC (Ping timeout: 369 seconds) [03:58] /dev/simfs 80G 35G 46G 43% / [03:58] will run out of disk 1st, but it may be expandable [04:00] *** chazchaz has joined #internetarchive.bak [04:03] * tpw_rules is currently downloading at around 10 floppies per second, or 1.3 CDs per minute [04:05] satas per fortnight [04:05] i calculate 1.3TB/day [04:06] 15MB/sec [04:08] what directories will be created in .git/annex that might contain data other than objects/ and bad/ [04:15] tmp/ [04:15] that's it? [04:15] yes [04:15] k, qool [04:15] if you mean large object data [04:15] yes [04:16] could there be a setting to just delete bad data instead of letting it hang around? [04:17] guess there could be.. [04:18] rm -rf will work find on that bad/ dir [04:57] oh btw, with today's git-annex release, you should be able to parallelize downloads better [04:57] try something like git annex get -j10 [04:58] er, -J10 [05:02] are you at all related to the iabak script itself? [05:07] will ./iabak auto-download the latest version of git-annex? [05:07] or do i have to remove that directory [05:09] it should auto-upgrade [05:09] 03registrar 05master 955c037 06other 10SHARD1/pubkeys registration of twatson52 on SHARD1 [05:10] 03registrar 05master 7b1b8a8 06other 10SHARD5/pubkeys registration of twatson52 on SHARD5 [05:10] 03registrar 05master e2a6e52 06other 10SHARD6/pubkeys registration of twatson52 on SHARD6 [05:10] what about automatically setting annex.sshcaching false? i think i've been missing registration because the ./checkoutshard script died [05:14] * tpw_rules thinks he is the whiniest member of this channel [05:43] woah, top of leaderboard maneuvering! :) [06:32] pikhq: I joined an anon chat room back in 2010 when they were doing their ddos's and bs'ed with them. FBI decided that made me a conspirator. [07:41] closure: how do I tell *which* of my shards is unregistered? :) [08:03] I like -J, but I miss seeing wget doing its thing :( [08:33] *** primus104 has joined #internetarchive.bak [08:46] woo -J is much faster than my hacky old multiple instances of iabak setup [10:05] *** Kenshin has quit IRC (Quit: ZNC - http://znc.in) [12:11] Senji: exactly :( [12:31] bah that figure for me is a lie. i don't have that much storage attached [12:36] Senji: slight mitigation: install iftop [13:14] *** primus104 has quit IRC (Leaving.) [14:32] *** VADemon has joined #internetarchive.bak [15:02] *** Kenshin has joined #internetarchive.bak [15:03] *** svchfoo2 sets mode: +o Kenshin [15:04] *** Atluxity has joined #internetarchive.bak [15:40] *** primus104 has joined #internetarchive.bak [16:42] *** primus104 has quit IRC (Leaving.) [17:24] *** VADemon_ has joined #internetarchive.bak [17:25] *** svchfoo3 has quit IRC (Read error: Connection reset by peer) [17:30] *** VADemon has quit IRC (west.us.hub irc.eversible.com) [18:13] *** primus104 has joined #internetarchive.bak [18:32] *** svchfoo3 has joined #internetarchive.bak [18:32] *** svchfoo2 sets mode: +o svchfoo3 [18:34] ryang: Oh "fun". [18:42] *** lhobas_ has joined #internetarchive.bak [18:56] *** primus104 has quit IRC (Leaving.) [19:11] *** kyan has joined #internetarchive.bak [19:32] *** primus104 has joined #internetarchive.bak [19:46] *** kyan has quit IRC (Quit: This computer has gone to sleep) [21:05] closure ryan.g is me :p [21:06] you guys for some reason wanted an email reg and not a nickname like every other project ;) [21:08] my box is now temp down since it lives in the cloud and my friend who operates that cloud is having some openstack issues. should be back by monday hopefully [21:28] The point of the email reg is to have a way to contact someone who has stuff expiring.