[02:25] *** sevs has quit IRC (Ping timeout: 268 seconds) [02:37] *** db48x has joined #internetarchive.bak [02:44] 03registrar 05master 6d67f2c 06other 10SHARD16/pubkeys registration of iabak on SHARD16 [02:53] *** db48x has quit IRC (Read error: Operation timed out) [03:26] *** db48x has joined #internetarchive.bak [03:47] *** VADemon has quit IRC (Quit: left4dead) [04:58] I got out of memory trying to git annex sync shard4, retrying to see if was a one time thing [05:05] 03registrar 05master 57ea6a6 06other 10SHARD18/pubkeys registration of iabak on SHARD18 [05:06] 03registrar 05master f607ae1 06other 10SHARD17/pubkeys registration of iabak on SHARD17 [05:34] probably why i was having slowness issues earlier, swapping [05:56] 03registrar 05master 8c0f49a 06other 10SHARD19/pubkeys registration of iabak on SHARD19 [06:07] thelsdj: ah, fun [06:07] thelsdj: how much memory does it need? [06:13] closure: I have an idea [06:13] git-annex fromkey can take input on stdin, which is nice [06:13] git-annex rekey can take the same input as paris of arguments, which is ok but somewhat annoying [06:14] but the real annoyance is that they don't work the same way [06:15] while trying to read the source for them I decided that what I really want is a git-annex command that takes a stream of commands on stdin and dispatches them to the other git-annex commands (so "rekey foo bar\nfromkey foo bar\n...") [06:50] *** Whopper has quit IRC (Remote host closed the connection) [07:31] *** kula has quit IRC (Read error: Operation timed out) [07:32] *** kula has joined #internetarchive.bak [07:42] 03registrar 05master 497d95a 06other 10SHARD4/pubkeys registration of iabak on SHARD4 [07:44] my device has ~850M available for programs, i have 2G swap as well, obviously if it fails again I could increase swap [07:44] i stopped some other services, the sync is still running so will see [07:44] its only using about 500M of swap so far [08:10] 03registrar 05master a44e197 06other 10SHARD11/pubkeys registration of iabak on SHARD11 [08:19] 03registrar 05master f78b961 06other 10SHARD3/pubkeys registration of iabak on SHARD3 [08:28] 03registrar 05master 3389840 06other 10SHARD10/pubkeys registration of iabak on SHARD10 [08:30] 03registrar 05master aa142f7 06other 10SHARD12/pubkeys registration of frogging-iabak on SHARD12 [08:30] 03registrar 05master 76a0ed2 06other 10SHARD4/pubkeys registration of frogging-iabak on SHARD4 [08:34] 03registrar 05master d9bbb23 06other 10SHARD18/pubkeys registration of iabak on SHARD18 [08:35] 03registrar 05master 89c852e 06other 10SHARD19/pubkeys registration of iabak on SHARD19 [08:35] 03registrar 05master f5fa722 06other 10SHARD17/pubkeys registration of iabak on SHARD17 [08:46] *** kula has quit IRC (Read error: Operation timed out) [08:47] *** kula has joined #internetarchive.bak [09:59] thelsdj: so about 1.5GB total? [10:24] 03registrar 05master cc4d7e1 06other 10SHARD17/pubkeys registration of deewiant+ia.bak on SHARD17 [10:44] *** bwn has quit IRC (Read error: Operation timed out) [10:51] nooooo! [10:51] search for items: 18.2k 0:00:07 [2.44k/s] [10:51] enumerate files: 137k 4:51:10 [ 0 /s]jq: error (at :11283): Cannot iterate over null (null) [10:51] jq: error (at :11284): Cannot iterate over null (null) [10:54] *** bwn has joined #internetarchive.bak [10:58] oh, huh. it's still running [11:36] *** sevs has joined #internetarchive.bak [12:00] 03registrar 05master 52b6873 06other 10SHARD16/pubkeys registration of email on SHARD16 [12:06] *** antomati_ is now known as antomatic [12:23] *** bohdans has joined #internetarchive.bak [12:24] Hi all, I have followed the instructions (clone git, run iabak), but I am unsure on how to register the client? [12:32] bohdans: the first time you run iabak it prompts you to enter your email address [12:32] it then uses that email address to register [12:32] bohdans: does your email address start with "email@"? [12:36] db48x, does iabak stop all downloads until everything that has downloaded has checksummed? [12:37] HCross: when you start iabak and it finds that you already have some shards, it first runs git-annex fsck on them before it downloads anything new [12:38] so yea, you could say that it's not downloading anything while it checks the checksums [12:40] @db48x, yes [12:40] db48x, ah, on this piddily ARM processor it takes a while [12:41] thats me, I am guessing my SPAM filter has grey listed, i'll wait a while and see if a email comes through [12:41] bohdans: excellent. half an hour ago we got this notification: [12:41] [07:00] registrar master 52b6873 other SHARD16/pubkeys registration of email on SHARD16 [12:41] [12:41] bohdans: there's no registration email to wait for [12:42] HCross: ah [12:42] cool :) [12:42] so this must be me, 2f28e312bc63b5b8399ab3ea85aa901b1176c72c email 0TB [12:43] yep :) [12:43] If i had known that, I would of used a cooler email address lol [12:43] heh [12:44] I have been reading the site, but I can't seem to find a est. of how much space this project needs? [12:44] you can change it if you like, just run the change-email script [12:44] we need about 20PB [12:44] give or take [12:45] wow ok, so 82 TB is just a drop then [12:45] 82 TB is the largest drop we will have :D [12:46] 03registrar 05master 877dbbb 06other 10SHARD16/pubkeys registration of bohdans on SHARD16 [12:47] 03registrar 05master 5fc8cb5 06other 10SHARD5/pubkeys registration of jdamery+iabak on SHARD5 [12:47] db48x: thank you, its midnight here so I'm going to let this sync overnight and see how it goes. Will look at brining another node up tomorrow. [12:47] bohdans: you're welcome [12:48] bohdans: feel free to ask as many questions as you like :) [12:50] 5.5TB left to go on this server, had to reboot it as it stopped [13:58] *** bohdans has quit IRC (Read error: Connection reset by peer) [13:59] *** bohdans has joined #internetarchive.bak [14:30] 03registrar 05master daf36b2 06other 10SHARD12/pubkeys registration of bohdans on SHARD12 [15:14] 03registrar 05master 9183e65 06other 10SHARD10/pubkeys registration of rbx1test on SHARD10 [15:30] 03registrar 05master ed2e539 06other 10SHARD11/pubkeys registration of rbx1test on SHARD11 [15:37] 03registrar 05master 7ac07e7 06other 10SHARD14/pubkeys registration of rbx1test on SHARD14 [16:05] 03registrar 05master c6dbb41 06other 10SHARD16/pubkeys registration of rbx1test on SHARD16 [17:30] *** sep332 has joined #internetarchive.bak [18:17] db48x: i'm at ~750M swap used and its still running so yeah i'm probably around 1.5G right now, but may be more before its done [18:22] 03registrar 05master 5599195 06other 10SHARD12/pubkeys registration of rbx1test on SHARD12 [18:23] another problem is that i think the cronjob is also trying to sync the same shard, so may be that what is using too much ram is 2 copies of trying to sync shard4 [18:28] maybe the flock is supposed to do that and i'm screwing it up by trying to run the sync manually [18:42] i really need a way to make the cronjob not run when i'm running iabak manually because i just don't seem to have enough ram for 2 git annex syncs at the same time regardless of which shard it is [18:43] run iabak manually, wait for it to add the cronjob [18:43] then remove it [18:43] yeah that makes sense [19:57] making really good use of this unlimited usage at home :p [20:20] 03registrar 05master af195a3 06other 10SHARD3/pubkeys registration of rbx1test on SHARD3 [20:30] 03registrar 05master 0c2eb7a 06other 10SHARD15/pubkeys registration of rbx1test on SHARD15 [22:35] If anyone's interested, Hetzner serverboerse have cheap 7x1.5TB servers for under 40eur [22:44] 03registrar 05master 7bb07c3 06other 10SHARD10/pubkeys registration of milenko on SHARD10 [22:55] *** sevs has quit IRC (Ping timeout: 268 seconds)