#internetarchive.bak 2017-03-09,Thu

↑back Search

Time Nickname Message
01:14 🔗 iabak-reg 03registrar 05master f613933 06other 10SHARD23/pubkeys registration of hcross on SHARD23
03:25 🔗 iabak-reg 03registrar 05master a812df2 06other 10SHARD16/pubkeys registration of hcross on SHARD16
03:34 🔗 kyan has quit IRC (Quit: Leaving)
03:34 🔗 kyan has joined #internetarchive.bak
04:51 🔗 iabak-reg 03registrar 05master 1388fb8 06other 10SHARD21/pubkeys registration of hcross on SHARD21
04:58 🔗 iabak-reg 03registrar 05master e82c691 06other 10SHARD18/pubkeys registration of hcross on SHARD18
07:21 🔗 svchfoo3 has quit IRC (Remote host closed the connection)
07:23 🔗 svchfoo3 has joined #internetarchive.bak
07:24 🔗 svchfoo1 sets mode: +o svchfoo3
08:04 🔗 iabak-reg 03registrar 05master 7d03fe6 06other 10SHARD14/pubkeys registration of hcross on SHARD14
08:22 🔗 iabak-reg 03registrar 05master 98dde50 06other 10SHARD20/pubkeys registration of hcross on SHARD20
08:29 🔗 atomotic has joined #internetarchive.bak
09:32 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
09:39 🔗 iabak-reg 03registrar 05master ce230bf 06other 10SHARD15/pubkeys registration of hcross on SHARD15
10:17 🔗 atomotic has joined #internetarchive.bak
11:28 🔗 iabak-reg 03registrar 05master bcce85f 06other 10SHARD10/pubkeys registration of hcross on SHARD10
11:40 🔗 iabak-reg 03registrar 05master d4646f5 06other 10SHARD6/pubkeys registration of hcross on SHARD6
12:59 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
14:37 🔗 atomotic has joined #internetarchive.bak
15:09 🔗 jmeyer has joined #internetarchive.bak
15:26 🔗 jmeyer hey, all. I've been running the client for a couple months now, and have now received my second "expiry warning" email, despite client running continuously. Any thoughts? Does the client only "check in" at startup?
15:27 🔗 jmeyer When I force-stop the client and restart it, the webpage status updates within a day, but I get another warning, weeks later.
15:28 🔗 db48x has quit IRC (Read error: Operation timed out)
15:29 🔗 db48x has joined #internetarchive.bak
15:30 🔗 Jon don't know the answer jmeyer but I basically have the same problem(s)
15:31 🔗 Jon I have parts of two shards but I think somehow they're registered to different UUIDs
15:31 🔗 Jon and running ./iabak only ever updates one of them
15:32 🔗 Senji Do you have the cronjob/systemd-thing installed? (It should have installed automatically, but...)
15:32 🔗 Senji I think I've heard of the cronjob not running if the downloader is going too
15:38 🔗 jmeyer Jon: I also have parts of two shards (don't know if relevant), but if running ./iabak only ever updates one, shouldn't that mean the other has reached 1+3 copies?
15:41 🔗 jmeyer Senji: I believe so, at least when I restart it tells me "You've already got the iabak-cronjob installed. I didn't have to do anything." or some such, so maybe the latter case you mentioned is happening.
15:42 🔗 db48x jmeyer: "checking in" with the server is just a matter of running "git annex sync" every now and then. both the client and the cronjob do this, or you can do it manually
15:42 🔗 db48x jmeyer: check your logs to see if the cron job (or systemd service, as appropriate for your system) has actually been running
15:43 🔗 db48x Jon: run "git annex info" in a shard to see what UUID it is
15:43 🔗 Senji db48x: the client will only sync on the shard it's working on though, won't it?
15:43 🔗 db48x Senji: yes
15:43 🔗 db48x well
15:43 🔗 db48x when it starts up it syncs every shard it can find, then as it works on a shard it periodically syncs it
15:47 🔗 db48x Senji: in the short term, the email should have identified a shard repository by UUID. you can just run "git annex sync" in that repository to update it while you figure out what the problem is
15:50 🔗 jmeyer db48x: will do. thanks
15:51 🔗 db48x you're welcome
15:55 🔗 Jon db48x: thanks for the hint. that failed wth (No space left on device)
15:55 🔗 Jon at least for one of my shards. it's working for another. I seem to have 3 now.
15:56 🔗 Jon my layout is to have separate filesystems for ~iabak (within which the git clone, local git-annex copy etc.) -- not full; ~iabak/shard3 (full); ~iabak/shard4 (not full)
15:56 🔗 Jon this is forced upon me somewhat by my local HDD arrangement, I'm dedicating 1T from one drive and 1T from another
15:57 🔗 Jon wow git annex info takes a while to run. still waiting for 'local annex keys:'
15:57 🔗 Senji If the disk is full then sync won't work; it needs about 0.5-2GB to run depending on the shard
15:58 🔗 Jon why has it filled the disk up then? shouldn't it limit itself to available space?
15:58 🔗 Jon or alternatively, can I get it to use another location for its working directory?
15:59 🔗 Jon 'git annex status' in my third shard exits with no output and a success status code (filesystem is not full)
15:59 🔗 Senji There is a configurable for the amount of disk space to leave empty at the end. I thought it was set to a sensible value by default but I don't know.
15:59 🔗 db48x status tells you whether you have any changes to commit
15:59 🔗 db48x Senji: that's only used by iabak, not by git-annex directly
16:00 🔗 db48x or rather, it's only used by git-annex when downloading files (such as when iabak is running)
16:00 🔗 db48x if you don't leave room for git metadata, it can grow through that extra space
16:00 🔗 Senji db48x: yes, but git-annex directly won't fill up the disk except for neeing a bit of space to make sync work
16:01 🔗 Jon I imagine if I zapped all this I could start over and get something non-broken, but I'd rather not throw away the 1.1T I've already got (looking , seems I have 1T of shard3 = full filesystem, and only 200G of shard4 which is seems to never try to fetch more for)
16:02 🔗 db48x Jon: to free up some space, just use 'git annex drop' on a few things
16:02 🔗 Jon OK, thanks
16:04 🔗 jmeyer Jon: can the client even automatically use/ manage multiple hardrives/ separate filesystems?
16:04 🔗 Jon jmeyer: honestly I don't know. I figured if it was fetching multiple shards, it would "just work" if the shard subdirs were on separate filesystems, but idk
16:04 🔗 Jon I guess I could run completely separate instances on each filesystem. not sure I could share a UUID in that situation
16:04 🔗 jmeyer Jon: or do you mean you have 2*1TB married JBOD?
16:05 🔗 db48x Jon: you could also run 'git annex find --copies=5' to find files that we have a lot of copies of, and drop those
16:05 🔗 Senji I have apparently 32 different shards; mostly one per fileystem across three computers.
16:07 🔗 Jon hmm git-annex drop won't function either. but movign/symlinking .git/annex/misctmp to ~ (with space) seems to help
16:07 🔗 Jon jmeyer: no they are distinct. I'd rather avoid JBODding them since that increases likelyhood of failure
16:07 🔗 Jon db48x: that's a cool idae
16:08 🔗 db48x .git/annex/misctmp is usually empty
16:08 🔗 Jon "Warning: This repository is currently marked as dead." aha that's useful to know
16:08 🔗 db48x I think that's where it stores incomplete downloads
16:08 🔗 Jon db48x: it seems to be where it puts working files to run 'git annex drop' , 'git annex info' etc. -- both failed until I symlinked it to a FS with space
16:08 🔗 db48x ah, I see
16:09 🔗 Jon OK so I know now that my shard3 copy (1T) is marked dead.
16:09 🔗 db48x Jon: run 'git annex semitrust <uuid>' to mark it as non-dead
16:09 🔗 db48x then fsk it
16:09 🔗 db48x fsck
16:09 🔗 Jon OK
16:10 🔗 db48x and then sync it, of course
16:10 🔗 Jon ah slight problem. my UUID is not output in the initial part of the 'git annex info' output, unless it's the UUID with no descriptive suffix (for my shard4, it was clearly labelled iabak@phobos:~/IA.BAK/shard4 [here])
16:12 🔗 Jon oh git config -l shows it
16:13 🔗 db48x you beat me to it :)
16:14 🔗 db48x Jon: btw, is this you? http://iabak.archiveteam.org/client/eb34830fd1da871513693e2e6c9ca4a09e69babc.html
16:16 🔗 Senji db48x: no, that's me
16:17 🔗 db48x ah :)
16:17 🔗 Senji You know something when you recognise your own UUID... :)
16:17 🔗 db48x heh
16:18 🔗 db48x Senji: a lot of yours need syncing as well
16:19 🔗 Senji Yeah, the hardware hooking the drives up to oklina is a bit ropey and occasionally needs poking.
16:20 🔗 db48x as long as we're all here, I'd like to sync some of our repositories directly, so that we can catch up on some files that have gone dark
16:20 🔗 db48x Jon, Senji, jmeyer: can we exchange ssh access?
16:23 🔗 db48x my ssh key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJQkqIgZ7D8WHW5Y3o+fpZC/4xtv/3IQrORJrTPCt7KY db48x@erebor
16:24 🔗 Senji Hmm, most of mine are behind NAT (and my bandwidth is metered except for a 4 hour window overnight, which might present a problem) :)
16:24 🔗 db48x hmm
16:24 🔗 db48x Senji: well, I could give you access to my machine, and you could push files to it
16:25 🔗 Senji Oh, and cleopatra's sshd doesn't support ed25519 :-D
16:25 🔗 Senji I could do that
16:26 🔗 db48x bah, one sec
16:27 🔗 db48x ssh-rsa
16:27 🔗 db48x AAAAB3NzaC1yc2EAAAADAQABAAAEAQC7/PISfISdeRw/cVWgszHu9Cy/Pz+T8r6zQYqZaDqkRTp4qLOHXbcwYVgNEXGQSq7f97qNQC2gv1Mae0JfArocmIrFcN5vlLoKovaemvBjk86tvDKwlcI5ZO3neWWBKN9dwdju1KvsAO/mwOa0QxnZjKz47CCOVKls1SnjdydEMmFLTMoArb8Qn/eAxv3wCHGFNpqYU4ErZftVfE8jzIb2YzX67zWfymEI0niuBNgvJ/lylCq0e8/Vdf5w2fHLxmBSwc637FdZZxE4pqEgnA2Qny73IBL3+U4ugETsdONYtsE/QiQotMcRosLHHU863kYCXTezVd10nhzUx3xrPyXV1Rir9jzqOkn10sAHMT0/0e/x3uDBrgLa/tI3eUgPerSVvnnWy4v
16:27 🔗 db48x db48x@erebor
16:30 🔗 Senji That key should now have access to jdamery-iabak@cleopatra.thy.me.uk (which machine has passphraseless login to the other two)
16:31 🔗 db48x send me your ssh key as well
16:32 🔗 Jon I've got to run, but I'll take a look later
16:33 🔗 db48x Jon: ok :)
16:33 🔗 Senji ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAut2lpYEg71g+dZaONUJxNFJ5mEaAx33aq5u9TiCCBPuorZo9L/Qiwe187txKExolkviVCJipTADNIPTBK05akRQAqTqqG80WodO7Ul5lSsrz0C4Acvn56jMRkij0a34b2BELhjFyKYKSiHg9n9vwkOJ/yRaLWTNUJuej9lxM2UFvgnQ== jdamery-iabak@cleopatra
16:33 🔗 Senji if you're happy with dodgy government elpitic curves :)
16:34 🔗 db48x for this purpose I think it will be fine
16:36 🔗 db48x Senji: ok, you should be able to log in as senji@erebor.db48x.net
16:37 🔗 Senji It's not accepting that key
16:39 🔗 db48x what's your ip address?
16:40 🔗 Senji cleopatra.thy.me.uk has address 81.187.132.36
16:40 🔗 Senji cleopatra.thy.me.uk has IPv6 address 2001:8b0:3b9:1:20a:cdff:fe22:8a5d
16:41 🔗 db48x curious...
16:42 🔗 db48x oh, right! port 23 :)
16:43 🔗 Senji "no hostkey alg"
16:43 🔗 Senji I've never seen that error before
16:44 🔗 db48x it means you have a super-old ssh, and I've turned off the super-old key types
16:45 🔗 Senji Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2
16:45 🔗 Senji I knew I had an old ssh though, see the comments about ed25519 :)
16:46 🔗 db48x 6.0, sheesh
16:47 🔗 jmeyer db48x: what would git-annex key exchange help with?
16:48 🔗 Senji db48x: yeah, upgrades are on my todo list :(
16:48 🔗 Senji I hate todo lists
16:48 🔗 db48x Senji: ok, I told it to advertise an ecdsa key, which your ssh client should accept (in addition to an ed25519 key)
16:49 🔗 db48x jmeyer: files on IA occasionally go dark, and are no longer available
16:49 🔗 db48x jmeyer: which means that git-annex cannot download them
16:49 🔗 db48x jmeyer: if we allow each other access to our machines via ssh, we can tell git annex to download files directly from each other
16:50 🔗 db48x jmeyer: we would use this to make additional copies of files not currently available from the web, as needed
16:50 🔗 Senji Now it's managing host key acceptance; but your server is still not accepting my ecdsa key
16:50 🔗 db48x hmm
16:52 🔗 jmeyer db48x: ah ,gotcha. while this would be cool, i'm still a newb to SSH to be fully confident in my security skills :-/ if I can spend some time in the next week or two square some things away on this machine, I'll msg you back....but no promises with 2 kids.
16:52 🔗 db48x jmeyer: fair enough :)
16:54 🔗 db48x Senji: try again?
16:55 🔗 Senji No luck
16:55 🔗 Senji Sorry if I'm a bit slow; I'm talking to tech support about a work problem at the same time :)
16:55 🔗 Senji (work's systems don't even support ECDSA, so they're even older...)
16:56 🔗 jmeyer db48x: also, thank you for the log pointer, it would appear when I moved my iabak storage pool, I borked the systemd timer. I though I had made the move cleanly.
16:56 🔗 db48x jmeyer: you're welcome :)
16:57 🔗 jmeyer been a gnu/linux user for 15 years...still a newbie in so many ways
16:57 🔗 db48x Senji: ok, try it again. it doesn't really say why it's rejecting it, but I can confirm that you're presenting the key that I added to your authorized_keys file
16:59 🔗 Senji I'm getting Connection refused now
16:59 🔗 Senji Debugging these kinds of ssh authentication issues is a real pain; I've been on the other end of that :(
17:00 🔗 db48x odd
17:00 🔗 db48x daemon is running...
17:02 🔗 Senji firewall?
17:03 🔗 db48x the NAT is NATting
17:12 🔗 db48x oh, right
17:13 🔗 db48x https://hastebin.com/yowunupuna :)
17:15 🔗 db48x ok, I unbanned you
17:15 🔗 Senji aww
17:15 🔗 Senji Still doesn't work
17:15 🔗 db48x !
17:15 🔗 Senji Does get to presenting the key though
17:16 🔗 db48x ok, I turned up the log level. could you try it again?
17:17 🔗 Senji done
17:17 🔗 db48x well: "Mar 09 09:17:01 erebor sshd[622]: debug1: Could not open authorized keys '/home/senji/.ssh/authorized_keys': Permission denied"
17:17 🔗 db48x yea, selinux :)
17:17 🔗 db48x ok, 7th time's the charm
17:18 🔗 Senji no :(
17:20 🔗 db48x heh, minor details: sudo chown senji.senji ~senji/.ssh/authorized_keys
17:20 🔗 db48x not sure if that would break it or not
17:20 🔗 Senji :-D
17:20 🔗 Senji I've been locked out by systemd again
17:20 🔗 db48x you are unbanned again
17:21 🔗 db48x for some reason, fail2ban doesn't actually ignore your ip address
17:21 🔗 Senji [senji@erebor ~]$
17:21 🔗 Senji success
17:21 🔗 Senji Which is good because I have to go make dinner soon
17:21 🔗 db48x aha!
17:21 🔗 db48x cool. my shards are in ~db48x/archives/IA.BAK
17:22 🔗 db48x and I'm starting to think about breakfast
17:23 🔗 db48x you can use "git remote add db48x ..." in a shard, then "git annex copy --to=db48x --not --copies=4" or similar to copy files to mine
17:23 🔗 Senji And you have lots of free disk space
17:24 🔗 db48x some, yes
17:24 🔗 Senji Saying that just fried my brain; because in the other conversation I'm talking about petabytes of data...
17:24 🔗 db48x lol
17:24 🔗 db48x I wish I had a petabyte
17:24 🔗 Senji Professionally a terabyte is a small amount of data :)
17:24 🔗 Senji We have about 4 petabytes on tapes
17:25 🔗 Senji Sadly the costs of such storage is well beyond the IA
17:25 🔗 db48x that's a lot of tapes
17:27 🔗 Senji I'd better see if SWMBO is ready to do dinner yet. I'll look at pushing some data to you overnight my time (about 7 hours from now or so)
17:28 🔗 db48x sounds good to me
17:49 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
21:58 🔗 jmeyer has quit IRC (Quit: http://chat.efnet.org (Session timeout))
22:36 🔗 Senji db48x: while syncing:
22:36 🔗 Senji remote: error: insufficient permission for adding an object to repository database ./objects
22:36 🔗 Senji remote: fatal: failed to write object
22:36 🔗 Senji error: unpack failed: unpack-objects abnormal exit
22:37 🔗 Senji Did you get the same "explanation"?
22:37 🔗 Senji Bah, wrong channel. Sorry
22:37 🔗 Senji ignore the previous, but not the previous previous :)
22:55 🔗 db48x Senji: which shard?
22:57 🔗 Senji To erebor.db48x.net:~db48x/archives/IA.BAK/shard6
22:57 🔗 Senji which just happened to be the first one I looked at
22:59 🔗 db48x got it
22:59 🔗 db48x some of the files/directories had the wrong group
22:59 🔗 db48x fixed that, and made all the directories setgid again
23:01 🔗 Senji I'll try sync again. It's rather slow :)
23:02 🔗 Senji Hmm, same error again
23:03 🔗 Senji http://pastebin.com/gPwgAMuz
23:07 🔗 db48x oh, huh
23:07 🔗 db48x it didn't even have a .git/annex/objects
23:13 🔗 db48x must have been one that got created during testing, but where I never actually downloaded anything
23:13 🔗 Senji Ahh
23:13 🔗 Senji Yeah, you're not on the top-25 list
23:14 🔗 db48x we should report that as a bug
23:15 🔗 db48x closure: can't copy into a repository unless it's already downloaded something itself
23:16 🔗 db48x ok, I downloaded one item in every shard that didn't have an objects directory :)
23:16 🔗 Senji Hmm
23:17 🔗 Senji sync worked now but
23:17 🔗 Senji jdamery-iabak@cleopatra:/mnt/nasi/ia/IA.BAK/shard6$ PATH=../git-annex.linux/:$PATH git annex copy --to=db48x --not --copies=4
23:17 🔗 Senji git-annex: cannot determine uuid for db48x (perhaps you need to run "git annex sync"?) (remote.db48x.annex-ignore is set)
23:17 🔗 db48x hmm
23:17 🔗 db48x that's a new one
23:18 🔗 Senji I can see your uuid in 'git annex info'
23:19 🔗 db48x try setting remote.origin.annex-uuid to that value?
23:19 🔗 Senji Oh, but your line in 'git annex info' doesn't have the remote name listed
23:20 🔗 db48x hmm. it may be that git-annex-shell isn't in the path, if I'm reading this correctly
23:20 🔗 Senji Right, I 'git annex remove' and then 'git annex add'
23:20 🔗 Senji and it worked
23:21 🔗 Senji so presumably that was fixed when you fixed one of the intermediate errors but had left legacy stuff
23:21 🔗 db48x hrm
23:21 🔗 Senji I'm now pushing a number of small files to you
23:23 🔗 Senji I don't know if this is the best shard to be looking at if you don't have any files in it yet though
23:25 🔗 db48x it's got a pretty good set of files that are still IA-only, so that's a good choice
23:25 🔗 Senji If they're IA-only this isn't going to help :)
23:25 🔗 Senji It'll only help for files that are 1-copy
23:26 🔗 db48x true :)
23:27 🔗 db48x there's a decent chunk of those as well
23:28 🔗 db48x you might try -J 5 or something
23:28 🔗 Senji yeah.
23:30 🔗 Senji Hmm, I think I've managed to get myself firewalled out again with that.
23:32 🔗 Senji I think I want --fast; assuming you're not downloading on that shard at the time
23:34 🔗 db48x yep
23:34 🔗 db48x it should only ban you if you fail to log in several times in a row
23:37 🔗 Senji Yes, it just suddenly said that it couldn't access
23:39 🔗 db48x ok, I set it to ignore your ip
23:47 🔗 Senji I think this might be an application for ControlMaster
23:47 🔗 Senji but I'm not going to try setting that up right now
23:47 🔗 db48x git annex sets that up for you
23:48 🔗 Senji Hmm, then why is it apparently making so many connections.
23:48 🔗 Senji At the moment I'm waiting on sync again
23:49 🔗 Senji might have been faster just to not worry about re-transfering the files I'd already transfered
23:56 🔗 Senji OK, that looks to be copying at a reasonable rate; and I'm off to bed

irclogger-viewer