| Time |
Nickname |
Message |
|
02:25
🔗
|
|
sevs has quit IRC (Ping timeout: 268 seconds) |
|
02:37
🔗
|
|
db48x has joined #internetarchive.bak |
|
02:44
🔗
|
iabak-reg |
03registrar 05master 6d67f2c 06other 10SHARD16/pubkeys registration of iabak on SHARD16 |
|
02:53
🔗
|
|
db48x has quit IRC (Read error: Operation timed out) |
|
03:26
🔗
|
|
db48x has joined #internetarchive.bak |
|
03:47
🔗
|
|
VADemon has quit IRC (Quit: left4dead) |
|
04:58
🔗
|
thelsdj |
I got out of memory trying to git annex sync shard4, retrying to see if was a one time thing |
|
05:05
🔗
|
iabak-reg |
03registrar 05master 57ea6a6 06other 10SHARD18/pubkeys registration of iabak on SHARD18 |
|
05:06
🔗
|
iabak-reg |
03registrar 05master f607ae1 06other 10SHARD17/pubkeys registration of iabak on SHARD17 |
|
05:34
🔗
|
thelsdj |
probably why i was having slowness issues earlier, swapping |
|
05:56
🔗
|
iabak-reg |
03registrar 05master 8c0f49a 06other 10SHARD19/pubkeys registration of iabak on SHARD19 |
|
06:07
🔗
|
db48x |
thelsdj: ah, fun |
|
06:07
🔗
|
db48x |
thelsdj: how much memory does it need? |
|
06:13
🔗
|
db48x |
closure: I have an idea |
|
06:13
🔗
|
db48x |
git-annex fromkey can take input on stdin, which is nice |
|
06:13
🔗
|
db48x |
git-annex rekey can take the same input as paris of arguments, which is ok but somewhat annoying |
|
06:14
🔗
|
db48x |
but the real annoyance is that they don't work the same way |
|
06:15
🔗
|
db48x |
while trying to read the source for them I decided that what I really want is a git-annex command that takes a stream of commands on stdin and dispatches them to the other git-annex commands (so "rekey foo bar\nfromkey foo bar\n...") |
|
06:50
🔗
|
|
Whopper has quit IRC (Remote host closed the connection) |
|
07:31
🔗
|
|
kula has quit IRC (Read error: Operation timed out) |
|
07:32
🔗
|
|
kula has joined #internetarchive.bak |
|
07:42
🔗
|
iabak-reg |
03registrar 05master 497d95a 06other 10SHARD4/pubkeys registration of iabak on SHARD4 |
|
07:44
🔗
|
thelsdj |
my device has ~850M available for programs, i have 2G swap as well, obviously if it fails again I could increase swap |
|
07:44
🔗
|
thelsdj |
i stopped some other services, the sync is still running so will see |
|
07:44
🔗
|
thelsdj |
its only using about 500M of swap so far |
|
08:10
🔗
|
iabak-reg |
03registrar 05master a44e197 06other 10SHARD11/pubkeys registration of iabak on SHARD11 |
|
08:19
🔗
|
iabak-reg |
03registrar 05master f78b961 06other 10SHARD3/pubkeys registration of iabak on SHARD3 |
|
08:28
🔗
|
iabak-reg |
03registrar 05master 3389840 06other 10SHARD10/pubkeys registration of iabak on SHARD10 |
|
08:30
🔗
|
iabak-reg |
03registrar 05master aa142f7 06other 10SHARD12/pubkeys registration of frogging-iabak on SHARD12 |
|
08:30
🔗
|
iabak-reg |
03registrar 05master 76a0ed2 06other 10SHARD4/pubkeys registration of frogging-iabak on SHARD4 |
|
08:34
🔗
|
iabak-reg |
03registrar 05master d9bbb23 06other 10SHARD18/pubkeys registration of iabak on SHARD18 |
|
08:35
🔗
|
iabak-reg |
03registrar 05master 89c852e 06other 10SHARD19/pubkeys registration of iabak on SHARD19 |
|
08:35
🔗
|
iabak-reg |
03registrar 05master f5fa722 06other 10SHARD17/pubkeys registration of iabak on SHARD17 |
|
08:46
🔗
|
|
kula has quit IRC (Read error: Operation timed out) |
|
08:47
🔗
|
|
kula has joined #internetarchive.bak |
|
09:59
🔗
|
db48x |
thelsdj: so about 1.5GB total? |
|
10:24
🔗
|
iabak-reg |
03registrar 05master cc4d7e1 06other 10SHARD17/pubkeys registration of deewiant+ia.bak on SHARD17 |
|
10:44
🔗
|
|
bwn has quit IRC (Read error: Operation timed out) |
|
10:51
🔗
|
db48x |
nooooo! |
|
10:51
🔗
|
db48x |
search for items: 18.2k 0:00:07 [2.44k/s] |
|
10:51
🔗
|
db48x |
enumerate files: 137k 4:51:10 [ 0 /s]jq: error (at <stdin>:11283): Cannot iterate over null (null) |
|
10:51
🔗
|
db48x |
jq: error (at <stdin>:11284): Cannot iterate over null (null) |
|
10:54
🔗
|
|
bwn has joined #internetarchive.bak |
|
10:58
🔗
|
db48x |
oh, huh. it's still running |
|
11:36
🔗
|
|
sevs has joined #internetarchive.bak |
|
12:00
🔗
|
iabak-reg |
03registrar 05master 52b6873 06other 10SHARD16/pubkeys registration of email on SHARD16 |
|
12:06
🔗
|
|
antomati_ is now known as antomatic |
|
12:23
🔗
|
|
bohdans has joined #internetarchive.bak |
|
12:24
🔗
|
bohdans |
Hi all, I have followed the instructions (clone git, run iabak), but I am unsure on how to register the client? |
|
12:32
🔗
|
db48x |
bohdans: the first time you run iabak it prompts you to enter your email address |
|
12:32
🔗
|
db48x |
it then uses that email address to register |
|
12:32
🔗
|
db48x |
bohdans: does your email address start with "email@"? |
|
12:36
🔗
|
HCross |
db48x, does iabak stop all downloads until everything that has downloaded has checksummed? |
|
12:37
🔗
|
db48x |
HCross: when you start iabak and it finds that you already have some shards, it first runs git-annex fsck on them before it downloads anything new |
|
12:38
🔗
|
db48x |
so yea, you could say that it's not downloading anything while it checks the checksums |
|
12:40
🔗
|
bohdans |
@db48x, yes |
|
12:40
🔗
|
HCross |
db48x, ah, on this piddily ARM processor it takes a while |
|
12:41
🔗
|
bohdans |
thats me, I am guessing my SPAM filter has grey listed, i'll wait a while and see if a email comes through |
|
12:41
🔗
|
db48x |
bohdans: excellent. half an hour ago we got this notification: |
|
12:41
🔗
|
db48x |
[07:00] <iabak-reg> registrar master 52b6873 other SHARD16/pubkeys registration of email on SHARD16 |
|
12:41
🔗
|
db48x |
|
|
12:41
🔗
|
db48x |
bohdans: there's no registration email to wait for |
|
12:42
🔗
|
db48x |
HCross: ah |
|
12:42
🔗
|
bohdans |
cool :) |
|
12:42
🔗
|
bohdans |
so this must be me, 2f28e312bc63b5b8399ab3ea85aa901b1176c72c email 0TB |
|
12:43
🔗
|
db48x |
yep :) |
|
12:43
🔗
|
bohdans |
If i had known that, I would of used a cooler email address lol |
|
12:43
🔗
|
db48x |
heh |
|
12:44
🔗
|
bohdans |
I have been reading the site, but I can't seem to find a est. of how much space this project needs? |
|
12:44
🔗
|
db48x |
you can change it if you like, just run the change-email script |
|
12:44
🔗
|
db48x |
we need about 20PB |
|
12:44
🔗
|
db48x |
give or take |
|
12:45
🔗
|
bohdans |
wow ok, so 82 TB is just a drop then |
|
12:45
🔗
|
db48x |
82 TB is the largest drop we will have :D |
|
12:46
🔗
|
iabak-reg |
03registrar 05master 877dbbb 06other 10SHARD16/pubkeys registration of bohdans on SHARD16 |
|
12:47
🔗
|
iabak-reg |
03registrar 05master 5fc8cb5 06other 10SHARD5/pubkeys registration of jdamery+iabak on SHARD5 |
|
12:47
🔗
|
bohdans |
db48x: thank you, its midnight here so I'm going to let this sync overnight and see how it goes. Will look at brining another node up tomorrow. |
|
12:47
🔗
|
db48x |
bohdans: you're welcome |
|
12:48
🔗
|
db48x |
bohdans: feel free to ask as many questions as you like :) |
|
12:50
🔗
|
HCross |
5.5TB left to go on this server, had to reboot it as it stopped |
|
13:58
🔗
|
|
bohdans has quit IRC (Read error: Connection reset by peer) |
|
13:59
🔗
|
|
bohdans has joined #internetarchive.bak |
|
14:30
🔗
|
iabak-reg |
03registrar 05master daf36b2 06other 10SHARD12/pubkeys registration of bohdans on SHARD12 |
|
15:14
🔗
|
iabak-reg |
03registrar 05master 9183e65 06other 10SHARD10/pubkeys registration of rbx1test on SHARD10 |
|
15:30
🔗
|
iabak-reg |
03registrar 05master ed2e539 06other 10SHARD11/pubkeys registration of rbx1test on SHARD11 |
|
15:37
🔗
|
iabak-reg |
03registrar 05master 7ac07e7 06other 10SHARD14/pubkeys registration of rbx1test on SHARD14 |
|
16:05
🔗
|
iabak-reg |
03registrar 05master c6dbb41 06other 10SHARD16/pubkeys registration of rbx1test on SHARD16 |
|
17:30
🔗
|
|
sep332 has joined #internetarchive.bak |
|
18:17
🔗
|
thelsdj |
db48x: i'm at ~750M swap used and its still running so yeah i'm probably around 1.5G right now, but may be more before its done |
|
18:22
🔗
|
iabak-reg |
03registrar 05master 5599195 06other 10SHARD12/pubkeys registration of rbx1test on SHARD12 |
|
18:23
🔗
|
thelsdj |
another problem is that i think the cronjob is also trying to sync the same shard, so may be that what is using too much ram is 2 copies of trying to sync shard4 |
|
18:28
🔗
|
thelsdj |
maybe the flock is supposed to do that and i'm screwing it up by trying to run the sync manually |
|
18:42
🔗
|
thelsdj |
i really need a way to make the cronjob not run when i'm running iabak manually because i just don't seem to have enough ram for 2 git annex syncs at the same time regardless of which shard it is |
|
18:43
🔗
|
Kaz |
run iabak manually, wait for it to add the cronjob |
|
18:43
🔗
|
Kaz |
then remove it |
|
18:43
🔗
|
thelsdj |
yeah that makes sense |
|
19:57
🔗
|
Frogging |
making really good use of this unlimited usage at home :p |
|
20:20
🔗
|
iabak-reg |
03registrar 05master af195a3 06other 10SHARD3/pubkeys registration of rbx1test on SHARD3 |
|
20:30
🔗
|
iabak-reg |
03registrar 05master 0c2eb7a 06other 10SHARD15/pubkeys registration of rbx1test on SHARD15 |
|
22:35
🔗
|
HCross2 |
If anyone's interested, Hetzner serverboerse have cheap 7x1.5TB servers for under 40eur |
|
22:44
🔗
|
iabak-reg |
03registrar 05master 7bb07c3 06other 10SHARD10/pubkeys registration of milenko on SHARD10 |
|
22:55
🔗
|
|
sevs has quit IRC (Ping timeout: 268 seconds) |