#internetarchive.bak 2015-04-01,Wed

↑back Search

Time Nickname Message
00:27 πŸ”— patricko- is now known as patrickod
00:30 πŸ”— patrickod is now known as patricko-
01:03 πŸ”— patricko- is now known as patrickod
01:16 πŸ”— patrickod is now known as patricko-
02:26 πŸ”— closure db48x: maybe
02:28 πŸ”— db48x thanks :)
02:28 πŸ”— db48x https://pastebin.mozilla.org/8827827
02:29 πŸ”— db48x this is when I do a cabal build
02:32 πŸ”— closure hmm, that error message doesn't make a great deal of sense. It's complaining about database code generated by a macro
02:33 πŸ”— closure what version does this show? ghc-pkg list persistent
02:45 πŸ”— tpw_rules i'm still not getting very good speed
02:45 πŸ”— closure tpw_rules: one thing you could do is run multiple git-annex get in parallel
02:45 πŸ”— closure it'll parallellize fine
02:46 πŸ”— tpw_rules will they not munch over each other?
02:46 πŸ”— closure nope
02:46 πŸ”— closure be a bit faster if you run them on different directories, but even on the same one they'll cooperate
02:51 πŸ”— tpw_rules coolio. any other shards out yet?
02:53 πŸ”— tpw_rules 8 in parallel!
02:53 πŸ”— tpw_rules http://cl.ly/image/0F0G1T2F0c42/Screen%20Shot%202015-03-31%20at%209.52.34%20PM.png
03:00 πŸ”— closure -------------------------
03:00 πŸ”— closure ----------------------
03:00 πŸ”— closure remember to 'git annex sync' every now and then so we know what you have
03:00 πŸ”— closure also, I really want to put together some runner scripts in a repo, to handle this stuff
03:01 πŸ”— tpw_rules can it sync while it downloads?
03:01 πŸ”— closure yes, it can
03:01 πŸ”— closure every single git-annex command parallelizes with every other command, except for git annex assistant (and things will detect it andd refuse to run)
03:01 πŸ”— tpw_rules wow, that's really neat
03:02 πŸ”— closure it'd be neater if I had -jN built in, but I don't. On the plus side, I have an open ticket about it from $EMPLOYER :)
03:02 πŸ”— tpw_rules did you make it?
03:02 πŸ”— closure yes
03:04 πŸ”— tpw_rules how often is now and then? once a day?
03:05 πŸ”— closure sounds ok.. I was thinking if there's a script it could do it once per hour
03:05 πŸ”— tpw_rules i was gonna set it on a cronjob
03:09 πŸ”— tpw_rules is there a way to rate-limit all annexes running?
03:10 πŸ”— tpw_rules i'm running them all in one tmux session if that's a linux-specific thing
03:10 πŸ”— db48x closure:
03:10 πŸ”— db48x persistent-1.3.1.1
03:10 πŸ”— db48x persistent-2.0.8
03:13 πŸ”— closure hmm, my guess is the 2.0.8 one, I've never seen that one
03:14 πŸ”— closure try: cabal configure --constraint "persistent == 1.3.1.1"
03:28 πŸ”— db48x same error
03:29 πŸ”— SketchCow closure: Agreed on runner scripts.
03:29 πŸ”— db48x it's still loading 2.0.8
03:39 πŸ”— closure db48x: maybe try cabal update; cabal install persistent-2.1.2
03:44 πŸ”— db48x heh, same error and it's still loading 2.0.8
03:45 πŸ”— closure cabal configure -v2 or -v3 will tell you wby, but this gets hairy
03:45 πŸ”— closure probably need to update some other library too
03:46 πŸ”— db48x Dependency persistent ==2.0.8: using persistent-2.0.8
03:47 πŸ”— db48x doesn't say what it's a dependency of though...
03:48 πŸ”— db48x ah
03:49 πŸ”— db48x https://pastebin.mozilla.org/8827836
03:55 πŸ”— closure db48x: at this point, I'd just blow away ~/.cabal and ~/.ghc and start over ..
03:56 πŸ”— closure so, I have a runner script. Handles installing git-annex, setting up a dedicated ssh key, cloning repo, running downloads
03:59 πŸ”— closure can someone get me set up on the archiveteam github?
04:09 πŸ”— db48x closure: sure, what's your username on github?
04:10 πŸ”— db48x probably joeyh, unless that's an impersonator...
04:12 πŸ”— closure yeah, that'll do
04:12 πŸ”— closure can you do me a favor and make the repo too?
04:15 πŸ”— db48x what would you like to call it?
04:15 πŸ”— closure that's why I asked you to make it :)
04:15 πŸ”— db48x ah
04:16 πŸ”— db48x github suggests petulant-happiness
04:18 πŸ”— db48x closure: do you have any particular license in mind?
04:19 πŸ”— closure nah, whatever's typical for archiveteam scripts
04:26 πŸ”— db48x https://github.com/ArchiveTeam/IA.BAK
04:33 πŸ”— closure ERROR: Permission to ArchiveTeam/IA.BAK.git denied to joeyh.
04:33 πŸ”— closure hrm
04:33 πŸ”— chfoo i think a team needs to be created for it
04:34 πŸ”— db48x did you accept the invitation?
04:36 πŸ”— db48x sighs
04:36 πŸ”— db48x dist/build/git-annex/git-annex-tmp/Utility/Format.o: In function `sT2A_info':
04:36 πŸ”— db48x (.text+0x57e0): undefined reference to `shakespearezm2zi0zi0zi3_TextziHamlet_zdslookup1_info'
04:37 πŸ”— closure you probably need to cabal clean and restart build
04:38 πŸ”— closure ah here's the invite now
04:40 πŸ”— chfoo oh, never mind about the team. since it was an invite to be an admin, the team would be redundant
04:42 πŸ”— SketchCow Sweet
04:51 πŸ”— closure wow, it seems to work
04:58 πŸ”— closure ok, this is ready for someone else to try
04:58 πŸ”— closure https://github.com/ArchiveTeam/IA.BAK and have at it!
05:03 πŸ”— db48x finally:
05:03 πŸ”— db48x git-annex version: 5.20150331-gf842888
05:03 πŸ”— db48x Β ξ‚’Β anglachelΒ ξ‚°Β db48xΒ ξ‚°Β ~Β ξ‚±Β srcΒ ξ‚±Β git-annexΒ ξ‚°Β git annex version
05:06 πŸ”— closure db48x: about that persistent-2.0.8 version, was that shipped in some distribution or something?
05:06 πŸ”— db48x no clue
05:07 πŸ”— db48x hrm
05:07 πŸ”— db48x with celebdil and fanuidhol offline, I don't have much disk space...
05:13 πŸ”— closure ------------------------------------------------------------
05:14 πŸ”— closure If you're running git-annex manually, https://github.com/ArchiveTeam/IA.BAK has instructions to switch to using it
05:14 πŸ”— closure ------------------------------------------------------------
05:17 πŸ”— cloudyys (~cloudcake@[redacted]) has joined #internetarchive.bak
05:31 πŸ”— db48x closure: ------------------>8----------------------
05:31 πŸ”— db48x ------------------>8----------------------
05:31 πŸ”— db48x ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLutoB3Ru0W6MHBYBNVGzJZj8anW9eFfXjAkm1pk0+OLWexjuIbLyHZzXhW3umgXvTMazk7RDmAe79t18aEzWDb8/Irm8QiXxhaitp/OAM1yneQG414pCWIKmZLV+cvLGLEd+IUGsaowlvugNdUc7A6VlDZzm9+gpb9tojgFMDuKxrfuMd/HvS+AwGVAXhfuu3BWtD/hN1lQTR/6MtWM6DcyQkkg6ry3OE7KtOfqTcqvno3aXQnLKG9VNmWS0Si/1b80Wse/n2qGGsZGNp0UtBqRV+tFm/OzbziTELYUNfFyiwkLHiJTDsRZVINKGjQ3HI8rzhQjHbPU2YbYCquJRt db48x@anglachel
05:33 πŸ”— closure db48x: set up
05:34 πŸ”— db48x closure: thank you, kind sir!
05:36 πŸ”— csssuf that's a clever scissor emoticon
05:36 πŸ”— DFJustin ✁
05:37 πŸ”— SketchCow closure - Need a webpage soft link on your teamarchive1 account so it can be browsed for updates?
05:37 πŸ”— SketchCow Like some script that runs twice a day to see the updating of how many copies are out there?
05:38 πŸ”— SketchCow https://developers.google.com/chart/ has pretty, pretty charts
05:42 πŸ”— closure SketchCow: yeah, make that link and I'll put something there tomorrow
05:42 πŸ”— closure it'll probably be a json dump but ..
05:43 πŸ”— SketchCow https://google-developers.appspot.com/chart/interactive/docs/gallery/gauge I love this one and I have no idea where it'd be used.
05:46 πŸ”— SketchCow OK, html in your directory on teamarchive1
05:46 πŸ”— SketchCow Or do I have you elsewhere
05:46 πŸ”— closure teamarchive1 is fine
05:50 πŸ”— SketchCow closure - So it's possible to set up an area map we could use.
05:50 πŸ”— SketchCow Do you have the stats handy?
05:50 πŸ”— SketchCow Newest? i'll even take older stats.
05:51 πŸ”— db48x closure: https://github.com/ArchiveTeam/IA.BAK/pull/1
05:53 πŸ”— closure db48x: yeah, I just always installed git-annex rather than check version number
05:54 πŸ”— closure the script uses one feature that needs a rather new git-annex (from january or so)
05:59 πŸ”— SketchCow http://teamarchive1.fnf.archive.org/ia.bak/graph.html
05:59 πŸ”— SketchCow :)
06:00 πŸ”— db48x heh
06:01 πŸ”— closure numcopies +0: 68417
06:01 πŸ”— closure numcopies +1: 32991
06:01 πŸ”— closure numcopies +2: 1932
06:01 πŸ”— closure numcopies +3: 3
06:01 πŸ”— closure that's current stats
06:03 πŸ”— SketchCow Do a shift-reload on http://teamarchive1.fnf.archive.org/ia.bak/graph.html and boom
06:03 πŸ”— SketchCow How long do the stats take to generate?
06:03 πŸ”— closure a few minutes
06:05 πŸ”— closure with the new script, clients will be reporting every hour when they're downloading
06:05 πŸ”— closure so I'll just run it every half hour or so
06:06 πŸ”— SketchCow Is it somewhere on teamarchive1 that I can pull from it?
06:06 πŸ”— SketchCow Graph now has a little explanatory top.
06:10 πŸ”— closure ~closure/iabak-stats
06:10 πŸ”— closure pretend that's updating, but it won't until tomorrow..
06:18 πŸ”— bzc6p_ (~bzc6p@[redacted]) has joined #internetarchive.bak
06:20 πŸ”— bzc6p has quit (Read error: Operation timed out)
06:23 πŸ”— SketchCow OK, I wrote the script.
06:23 πŸ”— SketchCow it's sitting in there.
06:23 πŸ”— SketchCow graph-gen.sh
06:23 πŸ”— SketchCow I'm going to make it automatically run every 30 minutes.
06:24 πŸ”— GitHub17 (~GitHub17@[redacted]) has joined #internetarchive.bak
06:24 πŸ”— GitHub17 (~GitHub17@[redacted]) has left #internetarchive.bak
06:24 πŸ”— GitHub17 IA.BAK/public-pubic-typo 0e2c810 David Yip: Fix typo.
06:24 πŸ”— GitHub17 [IA.BAK] yipdw created public-pubic-typo (+1 new commit): http://git.io/jdLT
06:25 πŸ”— GitHub183 (~GitHub183@[redacted]) has joined #internetarchive.bak
06:25 πŸ”— GitHub183 (~GitHub183@[redacted]) has left #internetarchive.bak
06:25 πŸ”— GitHub183 [IA.BAK] yipdw opened pull request #2: Fix typo in iabak public key display (master...public-pubic-typo) http://git.io/jdLZ
06:25 πŸ”— closure hah, pubic keys
06:26 πŸ”— yipdw pubic keys protect private secrets
06:28 πŸ”— GitHub5 (~GitHub5@[redacted]) has joined #internetarchive.bak
06:28 πŸ”— GitHub5 (~GitHub5@[redacted]) has left #internetarchive.bak
06:28 πŸ”— GitHub78 (~GitHub78@[redacted]) has joined #internetarchive.bak
06:28 πŸ”— GitHub78 (~GitHub78@[redacted]) has left #internetarchive.bak
06:28 πŸ”— GitHub5 IA.BAK/master 82edd72 Joey Hess: Merge pull request #2 from ArchiveTeam/public-pubic-typo...
06:28 πŸ”— GitHub78 [IA.BAK] joeyh closed pull request #2: Fix typo in iabak public key display (master...public-pubic-typo) http://git.io/jdLZ
06:28 πŸ”— GitHub5 [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jdLd
06:29 πŸ”— yipdw sets mode -n #internetarchive.bak
06:32 πŸ”— SketchCow OK, crontab is in for graph.
06:34 πŸ”— yipdw agh, somehow I forgot that my storage pool runs on Solaris
06:34 πŸ”— yipdw this got a bit harder for me, will have to find some way
06:37 πŸ”— yipdw huh, I guess this works over CIFS
06:44 πŸ”— closure probably. However, disregard anything I said above about it supporting parallel operation when on a network filesystem..
06:48 πŸ”— yipdw yeah, that's fine
07:05 πŸ”— SketchCow (Idly checked... script that makes the graph is now properly running and updating.)
07:47 πŸ”— cloudyys has quit (Read error: Connection reset by peer)
10:00 πŸ”— hater anyone here who has write-access to https://github.com/ArchiveTeam/IA.BAK ?
10:01 πŸ”— hater i wrote a simple patch to check if all used tools (in the iabak-helper script) are available
10:01 πŸ”— hater http://pastebin.com/raw.php?i=EdNLPS05
10:12 πŸ”— db48x hater: :D
10:13 πŸ”— db48x hater: closure is probably asleep, but if you make a pull request he'll see it in the morning
10:13 πŸ”— hater oh, i forget the timeshift between countries every time..
10:16 πŸ”— db48x no worries
10:17 πŸ”— hater db48x: i knew i know your nick - i commented your pull request :D
10:17 πŸ”— db48x aha :)
10:31 πŸ”— GitHub59/#internetarchive.bak [IA.BAK] cancerAlot opened pull request #3: checkDeps should check if every tool is available (master...master) http://git.io/jF3E
10:31 πŸ”— hater nice bot
11:32 πŸ”— zottelbey (~zottelbey@[redacted]) has joined #internetarchive.bak
13:51 πŸ”— hater db48x: git-annex version: 5.20150327-g19a1a35 - is the g19a1a35 important?
14:15 πŸ”— sep332 hater: anything 5.20150205 or later should work
14:18 πŸ”— hater sep332: i'm writing a version-checker for the iabak-helper
14:19 πŸ”— hater if the g19a1a35-part is not important, the comparison will be much easier
14:19 πŸ”— closure that is the git rev, it's not important
14:20 πŸ”— hater ok, thx
15:05 πŸ”— GitHub97/#internetarchive.bak IA.BAK/server 0d67fac Joey Hess: add my server-side scripts
15:05 πŸ”— GitHub97/#internetarchive.bak IA.BAK/server 3cbe617 Joey Hess: ssh pub keys
15:05 πŸ”— GitHub97/#internetarchive.bak IA.BAK/server 8811b55 Joey Hess: move to server branch
15:05 πŸ”— GitHub97/#internetarchive.bak [IA.BAK] joeyh created server (+3 new commits): http://git.io/jNJb
15:05 πŸ”— GitHub142/#internetarchive.bak IA.BAK/master 0bf2f69 Joey Hess: moved to server branch
15:05 πŸ”— GitHub142/#internetarchive.bak IA.BAK/master 930a844 Joey Hess: Merge remote-tracking branch 'origin/master'
15:05 πŸ”— GitHub142/#internetarchive.bak [IA.BAK] joeyh pushed 2 new commits to master: http://git.io/jNUk
15:09 πŸ”— GitHub118/#internetarchive.bak [IA.BAK] joeyh created pubkey (+1 new commit): http://git.io/jNkT
15:09 πŸ”— GitHub118/#internetarchive.bak IA.BAK/pubkey ec50140 Joey Hess: move pubkey to its own branch...
15:09 πŸ”— GitHub149/#internetarchive.bak IA.BAK/server 7b41b4a Joey Hess: moved pubkeys to a separate branch
15:09 πŸ”— GitHub149/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNkk
15:24 πŸ”— GitHub137/#internetarchive.bak IA.BAK/server 37e9b37 Joey Hess: adding web hook for github repo
15:24 πŸ”— GitHub137/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNYw
15:26 πŸ”— GitHub3/#internetarchive.bak IA.BAK/server f248947 Joey Hess: cgi gunk
15:26 πŸ”— GitHub3/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNOd
15:29 πŸ”— svchfoo2 has quit (Read error: Connection reset by peer)
15:33 πŸ”— svchfoo2 (~chfoo2@[redacted]) has joined #internetarchive.bak
15:39 πŸ”— GitHub138/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNCK
15:39 πŸ”— GitHub138/#internetarchive.bak IA.BAK/server 4d86ca5 Joey Hess: finished web hook
15:40 πŸ”— GitHub94/#internetarchive.bak IA.BAK/server 031215b Joey Hess: allow seeing output
15:40 πŸ”— GitHub94/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNWU
15:42 πŸ”— GitHub99/#internetarchive.bak IA.BAK/server 8e2252a Joey Hess: auto-update SHARD1 authorized_keys
15:42 πŸ”— GitHub99/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNW5
15:43 πŸ”— GitHub26/#internetarchive.bak IA.BAK/server 60dd3f5 Joey Hess: typo
15:43 πŸ”— GitHub26/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNlY
15:44 πŸ”— GitHub105/#internetarchive.bak IA.BAK/server 6898ed3 Joey Hess: typo
15:44 πŸ”— GitHub105/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jN80
15:47 πŸ”— GitHub43/#internetarchive.bak IA.BAK/server e24c02e Joey Hess: fixes
15:47 πŸ”— GitHub43/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNBL
15:48 πŸ”— GitHub95/#internetarchive.bak IA.BAK/server f82b548 Joey Hess: bugfix
15:48 πŸ”— GitHub95/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNRv
15:50 πŸ”— closure ok, there's a pubkey branch in IA.BAK git repo now and anyone who can commit to it can add ssh public keys, which will auto-deploy to the server
15:51 πŸ”— closure also, the entire contents of the server is backed up on a couple git repos.. IA.BAK for the scripts and the clones of the shards themselves :)
15:51 πŸ”— closure self-backing up project ;)
15:52 πŸ”— SketchCow Are the stats generating yet? :)
16:01 πŸ”— closure almost there
16:03 πŸ”— closure I need a way to rsync a directory from Kenshin's server to teamarchive1
16:03 πŸ”— svchfoo1 has quit (Read error: Connection reset by peer)
16:04 πŸ”— closure or I could put them up on the web, but for the geo data, the IP addresses would need to be procesed first
16:05 πŸ”— svchfoo1 (~chfoo1@[redacted]) has joined #internetarchive.bak
16:06 πŸ”— GitHub34/#internetarchive.bak IA.BAK/pubkey 16146e9 Joey Hess: add Kazzy
16:06 πŸ”— GitHub34/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to pubkey: http://git.io/jNVr
16:06 πŸ”— Kazzy cheers
16:06 πŸ”— closure Kazzy: and nice point the people can pull-request to sign up! Although it might be better for someone to make a web site for registration, that just commits the pubkey
16:07 πŸ”— SketchCow We'll figure out geo in a bit, but the stats generating frequently is a (relatively easy) win.
16:07 πŸ”— SketchCow I figured out in theory how to use the IP list to make a really nice graph
16:08 πŸ”— SketchCow So that's there.
16:16 πŸ”— svchfoo1 has quit (Quit: Closing)
16:19 πŸ”— svchfoo2 has quit (Quit: Closing)
16:22 πŸ”— GitHub81/#internetarchive.bak IA.BAK/server 94399dc Joey Hess: stats gatherer
16:22 πŸ”— GitHub81/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNXM
16:24 πŸ”— GitHub11/#internetarchive.bak IA.BAK/server 3c7214f Joey Hess: fix dir name
16:24 πŸ”— GitHub11/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNMJ
16:25 πŸ”— GitHub126/#internetarchive.bak IA.BAK/server 79ca1db Joey Hess: typo
16:25 πŸ”— GitHub126/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jNMR
16:25 πŸ”— patricko- is now known as patrickod
16:28 πŸ”— svchfoo2 (~chfoo2@[redacted]) has joined #internetarchive.bak
16:30 πŸ”— patrickod is now known as patricko-
16:35 πŸ”— You've invited svchfoo3 to #internetarchive.bak (irc.teksavvy.ca)
16:35 πŸ”— svchfoo3 (~chfoo3@[redacted]) has joined #internetarchive.bak
16:35 πŸ”— chfoo gives channel operator status to svchfoo2 svchfoo3
16:41 πŸ”— GitHub94/#internetarchive.bak IA.BAK/server f481ac9 Joey Hess: set git commiter info (silly git..):
16:41 πŸ”— GitHub94/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jN7L
16:42 πŸ”— GitHub33/#internetarchive.bak IA.BAK/server c7f425b Joey Hess: update stats file atomically via tmp file
16:42 πŸ”— GitHub33/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jN7r
16:42 πŸ”— closure SketchCow: http://ia-bak.joeyh.name/stats/
16:43 πŸ”— GitHub27/#internetarchive.bak IA.BAK/server c20a5c8 Joey Hess: adjust to use newer apache html dir...
16:43 πŸ”— GitHub27/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to server: http://git.io/jN5U
16:43 πŸ”— closure still going up I see!
16:50 πŸ”— patricko- is now known as patrickod
16:52 πŸ”— GitHub75/#internetarchive.bak IA.BAK/server a4a8aab Joey Hess: add scripts I used to make SHARD1
16:52 πŸ”— GitHub75/#internetarchive.bak IA.BAK/server d3c6fa0 Joey Hess: script to setup a shard's git repo hooks
16:52 πŸ”— GitHub75/#internetarchive.bak [IA.BAK] joeyh pushed 2 new commits to server: http://git.io/jNNE
16:55 πŸ”— SketchCow closure: You want me to grab from there instead of the local file?
16:56 πŸ”— closure SketchCow: yes please
17:00 πŸ”— SketchCow OK, it's going. Thank you!
17:00 πŸ”— GitHub81/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jNjd
17:00 πŸ”— GitHub81/#internetarchive.bak IA.BAK/master b01ee4b Joey Hess: touch NOSHUF to disable the shuffling, if it's too expensive
17:01 πŸ”— SketchCow http://teamarchive1.fnf.archive.org/ia.bak/ will now update every 30 minutes from your script
17:01 πŸ”— closure btw, if we could get a better dns for the iabak server, something under archiveteam.org..
17:02 πŸ”— SketchCow What's the ip
17:03 πŸ”— svchfoo1 (~chfoo1@[redacted]) has joined #internetarchive.bak
17:03 πŸ”— svchfoo2 gives channel operator status to svchfoo1
17:04 πŸ”— closure 124.6.40.227
17:04 πŸ”— GitHub145/#internetarchive.bak IA.BAK/master a40453a Joey Hess: make sure we have the right git-annex installed even if shard repo already exists
17:04 πŸ”— GitHub145/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jAvF
17:05 πŸ”— SketchCow iabak.archiveteam.org now points there.
17:05 πŸ”— GitHub194/#internetarchive.bak IA.BAK/master 278f4e0 Joey Hess: fix path to NOSHUF
17:05 πŸ”— GitHub194/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jAfs
17:06 πŸ”— closure SketchCow: so, update your cron job to use the url http://iabak.archiveteam.org/stats/
17:07 πŸ”— GitHub32/#internetarchive.bak IA.BAK/master 2d3ff23 Joey Hess: use shiny new domain name
17:07 πŸ”— GitHub32/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jAJT
17:08 πŸ”— SketchCow Done
17:09 πŸ”— SketchCow So, just to put this in here, and I don't want to address it now
17:09 πŸ”— SketchCow IA co-worker asks what the potential is for a contributing user to also be a seed for the torrents for the items.
17:09 πŸ”— SketchCow I think it adds a lot of burden, I'd rather we stay on the current incremental path
17:10 πŸ”— closure well, they have the torrent files, and they have the contents of the torrent.. so they can run a torrent client
17:10 πŸ”— closure but it doesn't help the other ia.bak clients get it any quicker
17:10 πŸ”— SketchCow Sort of.
17:10 πŸ”— SketchCow they have SOME of them
17:11 πŸ”— closure yeah, they can git annex find some that they have
17:11 πŸ”— SketchCow No, no. This is literally to allow someone to download a hot item quicker
17:11 πŸ”— closure there are probably more torrents in a shard than can be managed by most torrent clients
17:13 πŸ”— SketchCow Agreed
17:13 πŸ”— closure goes away to implement distributed global fsck in git-annex
17:14 πŸ”— patrickod is now known as patricko-
17:39 πŸ”— patricko- is now known as patrickod
17:46 πŸ”— patrickod is now known as patricko-
18:33 πŸ”— midas closure: if the download speed is higher from torrents we could use it tho
18:34 πŸ”— sep332 maybe just for large files?
18:35 πŸ”— patricko- is now known as patrickod
18:35 πŸ”— patrickod is now known as patricko-
18:53 πŸ”— GitHub166/#internetarchive.bak [IA.BAK] db48x opened pull request #4: prefer downloading all files in an item together (master...items-together) http://git.io/jAQd
18:59 πŸ”— patricko- is now known as patrickod
19:02 πŸ”— GitHub193/#internetarchive.bak [IA.BAK] cancerAlot closed pull request #3: checkDeps should check if every tool is available (master...master) http://git.io/jF3E
19:02 πŸ”— bzc6p__ (~bzc6p@[redacted]) has joined #internetarchive.bak
19:05 πŸ”— SketchCow I haven't responded to DHSR's post on how IA.BAK is pretty impossible because too busy doing IA.BAK
19:05 πŸ”— SketchCow It's a tricky layer, just like encrypted is a tricky layer.
19:05 πŸ”— SketchCow drop mic
19:05 πŸ”— bzc6p_ has quit (Read error: Operation timed out)
19:16 πŸ”— wtron (4df54bfa@[redacted]) has joined #internetarchive.bak
19:16 πŸ”— wtron has quit (Connection closed)
19:17 πŸ”— yipdw buy my new book, Throwin' Shade With git-annex
19:27 πŸ”— patrickod is now known as patricko-
19:31 πŸ”— GitHub186/#internetarchive.bak IA.BAK/master c5306f5 Joey Hess: Merge pull request #4 from db48x/items-together...
19:31 πŸ”— GitHub186/#internetarchive.bak IA.BAK/master d3539eb Daniel Brooks: prefer downloading all files in an item together...
19:31 πŸ”— GitHub69/#internetarchive.bak [IA.BAK] joeyh closed pull request #4: prefer downloading all files in an item together (master...items-together) http://git.io/jAQd
19:31 πŸ”— GitHub186/#internetarchive.bak [IA.BAK] joeyh pushed 2 new commits to master: http://git.io/jxIL
19:35 πŸ”— GitHub198/#internetarchive.bak IA.BAK/pubkey 96bfe8f Joey Hess: zottelbey's key
19:35 πŸ”— GitHub198/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to pubkey: http://git.io/jxLU
19:50 πŸ”— patricko- is now known as patrickod
19:51 πŸ”— db48x oh yea, a 2GB blob of Ladies' Home Journal
19:53 πŸ”— patrickod is now known as patricko-
20:03 πŸ”— GitHub8/#internetarchive.bak [IA.BAK] cancerAlot opened pull request #5: don't reinstall git-annex everytime (master...updateLogic) http://git.io/jxWC
20:05 πŸ”— GitHub103/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jxll
20:05 πŸ”— GitHub103/#internetarchive.bak IA.BAK/master aa29021 Joey Hess: last pull request introduced a bug
20:08 πŸ”— db48x closure: what bug does that fix?
20:09 πŸ”— closure missing - in there
20:09 πŸ”— closure hater: I'm using tabs for indentation in the script, could you update your PR?
20:10 πŸ”— db48x yes, I see the change, but what is that going to do?
20:10 πŸ”— hater closure: damn, i configured vi to replace tabs with whitespaces
20:10 πŸ”— acridAxid has quit (Read error: Operation timed out)
20:11 πŸ”— db48x dirname - foo/bar gives two answers; "." and "foo"
20:11 πŸ”— db48x did you mean dirname --?
20:12 πŸ”— acridAxid (~acridAxid@[redacted]) has joined #internetarchive.bak
20:12 πŸ”— svchfoo2 gives channel operator status to acridAxid
20:13 πŸ”— closure well zottelbey said xargs needed a - to read stdin. Mine doesn't, but I guess his does
20:13 πŸ”— db48x dirname isn't reading stdin here
20:13 πŸ”— db48x xargs is reading stdin and feeding it to dirname on the command line
20:13 πŸ”— db48x echo foo/bar bar/baz | xargs dirname
20:14 πŸ”— zottelbey i got an error dirname: missing operand
20:14 πŸ”— closure oic, you mean the option is misplaced
20:14 πŸ”— db48x zottelbey: what does echo foo/bar bar/baz | xargs -t dirname print out?
20:15 πŸ”— zottelbey dirname foo/bar bar/baz
20:15 πŸ”— zottelbey foo
20:15 πŸ”— zottelbey bar
20:15 πŸ”— db48x that's what I expect...
20:15 πŸ”— zottelbey hm.
20:15 πŸ”— underscor has quit (Ping timeout: 370 seconds)
20:16 πŸ”— db48x does your dirname have any options besides --zero, --help and --version?
20:16 πŸ”— zottelbey nope.
20:16 πŸ”— db48x weird
20:16 πŸ”— underscor (~quassel@[redacted]) has joined #internetarchive.bak
20:16 πŸ”— db48x what version of dirname do you have?
20:17 πŸ”— zottelbey 8.23
20:17 πŸ”— zottelbey this is the whole printout for iabak. doesnt seem to be working for me at all btw: http://pastebin.com/eV3TMien
20:18 πŸ”— GitHub21/#internetarchive.bak IA.BAK/master 4812624 Joey Hess: ok, so that fix was wrong
20:18 πŸ”— GitHub21/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jxEl
20:18 πŸ”— db48x hmm, ok
20:18 πŸ”— hater closure: https://github.com/cancerAlot/IA.BAK/commit/bcfdefc3b8bee2d8a4d74899281e430c96aaf521
20:18 πŸ”— db48x zottelbey: can you modify your iabak-helper script so that the first line ends in -x?
20:18 πŸ”— db48x then when you rerun it it'll print out everything it does
20:19 πŸ”— closure so, I get "missing operand" when dirname is passed no parameter at all
20:19 πŸ”— zottelbey sure.
20:19 πŸ”— db48x aha: http://lingrok.org/xref/coreutils/src/dirname.c#116
20:19 πŸ”— closure so, xargs is running it on empty input
20:19 πŸ”— SketchCow has quit (Ping timeout: 255 seconds)
20:19 πŸ”— closure that can be fixed. But.. why is the input empty
20:19 πŸ”— zottelbey db48x, http://pastebin.com/jVe8dhML
20:20 πŸ”— db48x xargs -r, yea
20:21 πŸ”— closure zottelbey: does this find any files? git annex find --not --copies 4
20:21 πŸ”— GitHub43/#internetarchive.bak IA.BAK/master 3ad019c Joey Hess: pass --no-run-if-empty to xargs
20:21 πŸ”— GitHub43/#internetarchive.bak [IA.BAK] joeyh pushed 1 new commit to master: http://git.io/jxu2
20:21 πŸ”— zottelbey nope. no output at all
20:22 πŸ”— closure well, it finds stuff in my shard, so we're not actually done with the backup :)
20:22 πŸ”— zottelbey i would have doubted that in the first place
20:22 πŸ”— closure do you have a git-annex branch in the shard1 repo?
20:23 πŸ”— closure and you ran that in shard1, right?
20:23 πŸ”— zottelbey yes i did and yes, the folder shard1/.git/annex exists.
20:23 πŸ”— zottelbey if that is what you are asking
20:23 πŸ”— closure what does this say: git annex whereis internetarchivebooks/animalparasiteshe2chan/animalparasiteshe2chan_dc.xml
20:24 πŸ”— closure no, I mean does git branch command in shard1 show a git-annex branch
20:24 πŸ”— zottelbey yes it does
20:25 πŸ”— zottelbey "git-annex: internetarchivebooks/animalparasiteshe2chan/animalparasiteshe2chan_dc.xml not found"
20:25 πŸ”— GitHub198/#internetarchive.bak [IA.BAK] db48x opened pull request #6: don't let filenames get interpreted as command-line args (master...protect-against-filenames) http://git.io/jxzM
20:25 πŸ”— closure ok, that seems to be the problem..
20:25 πŸ”— closure what filesystem and OS is this repo on?
20:25 πŸ”— zottelbey ext4 on kubuntu
20:25 πŸ”— closure db48x: fwiw, git-annex does guard all pathnames to avoid args. I use the ./$file method
20:26 πŸ”— closure which works passing files to any program at all
20:26 πŸ”— closure zottelbey: does the shard1 repository have any branch checked out?
20:27 πŸ”— SketchCow (~jscott@[redacted]) has joined #internetarchive.bak
20:27 πŸ”— svchfoo1 gives channel operator status to SketchCow
20:28 πŸ”— db48x closure: hrm:
20:28 πŸ”— db48x internetarchivebooks/1910catholicencyclop07herb/1910catholicencyclop07herb_archive.torrent
20:28 πŸ”— db48x Β ξ‚’Β anglachelΒ ξ‚°Β db48xΒ ξ‚°Β ~Β ξ‚±Β projectsΒ ξ‚±Β IA.BAKΒ ξ‚±Β shard1Β ξ‚°Β git annex find --not --copies 4 | head
20:28 πŸ”— db48x internetarchivebooks/1910catholicencyclop07herb/1910catholicencyclop07herb_dc.xml
20:28 πŸ”— closure db48x: your pull request should also pass -- to git-annex get
20:28 πŸ”— closure that's what I see too with the command
20:29 πŸ”— db48x pushed
20:35 πŸ”— closure db48x: can you merge stuff on github? I'm out of time and BW
20:35 πŸ”— db48x sure
20:35 πŸ”— closure I think both these pull reqs are good
20:36 πŸ”— GitHub197/#internetarchive.bak IA.BAK/master 2467688 Daniel Brooks: don't let filenames get interpreted as command-line args...
20:36 πŸ”— GitHub197/#internetarchive.bak IA.BAK/master 4efee24 Daniel Brooks: Merge pull request #6 from db48x/protect-against-filenames...
20:36 πŸ”— GitHub197/#internetarchive.bak IA.BAK/master 576344b Daniel Brooks: also protect git annex get against filenames that look like args
20:36 πŸ”— GitHub118/#internetarchive.bak [IA.BAK] db48x closed pull request #6: don't let filenames get interpreted as command-line args (master...protect-against-filenames) http://git.io/jxzM
20:36 πŸ”— GitHub197/#internetarchive.bak [IA.BAK] db48x pushed 3 new commits to master: http://git.io/jxwS
20:37 πŸ”— zottelbey ill get back to this tomorrow. have to leave now.
20:38 πŸ”— GitHub183/#internetarchive.bak [IA.BAK] db48x closed pull request #5: don't reinstall git-annex everytime (master...updateLogic) http://git.io/jxWC
20:38 πŸ”— GitHub104/#internetarchive.bak IA.BAK/master 56ef85f Daniel Brooks: Merge pull request #5 from cancerAlot/updateLogic...
20:38 πŸ”— GitHub104/#internetarchive.bak IA.BAK/master 8fc19b2 cancerAlot: don't reinstall git-annex everytime
20:38 πŸ”— GitHub104/#internetarchive.bak IA.BAK/master bcfdefc cancerAlot: now with tabs
20:38 πŸ”— GitHub104/#internetarchive.bak [IA.BAK] db48x pushed 3 new commits to master: http://git.io/jxrz
20:38 πŸ”— db48x zottelbey: thanks!
20:42 πŸ”— zottelbey has quit (Remote host closed the connection)
20:50 πŸ”— bzc6p__ is now known as bzc6p
21:29 πŸ”— londoncal (~londoncal@[redacted]) has joined #internetarchive.bak
21:55 πŸ”— closure http://git-annex.branchable.com/devblog/day_270__distributed_fsck/
22:11 πŸ”— hatsefla1 has quit (Read error: Operation timed out)
22:52 πŸ”— GitHub122/#internetarchive.bak [IA.BAK] cancerAlot opened pull request #8: install the right arch of git-annex (master...master) http://git.io/jpBr
23:26 πŸ”— Kenshin has quit (Ping timeout: 246 seconds)
23:34 πŸ”— Kenshin (~rurouni@[redacted]) has joined #internetarchive.bak
23:34 πŸ”— svchfoo3 gives channel operator status to Kenshin
23:48 πŸ”— patricko- is now known as patrickod
23:49 πŸ”— tpw_rules closure: is there a way for git-annex to show the total size of everything annexed
23:49 πŸ”— tpw_rules anyway, i've downloaded 347 gigs so far
23:51 πŸ”— patrickod is now known as patricko-
23:53 πŸ”— SketchCow closure: There's been no change in http://iabak.archiveteam.org/stats/SHARD1
23:53 πŸ”— SketchCow in something like 9 hours.
23:57 πŸ”— tpw_rules closure: is there a way to rate-limit all the running git-annexes?
irclogger-viewer