| Time |
Nickname |
Message |
|
00:17
🔗
|
|
niyaje5 has joined #internetarchive.bak |
|
00:21
🔗
|
SketchCow |
Git Annex likes pie |
|
00:29
🔗
|
SketchCow |
Graphs have gone flat |
|
00:32
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
00:58
🔗
|
iabak-reg |
03registrar 05master 33fd59e 06other 10SHARD4/pubkeys registration of chosenken on SHARD4 |
|
00:59
🔗
|
iabak-reg |
03registrar 05master c78e515 06other 10SHARD6/pubkeys registration of chosenken on SHARD6 |
|
01:01
🔗
|
iabak-reg |
03registrar 05master 11a5dc5 06other 10SHARD4/pubkeys registration of alex.buie on SHARD4 |
|
01:21
🔗
|
SketchCow |
closure: Let me know about the possibility of adding low-size shards |
|
01:22
🔗
|
iabak-reg |
03registrar 05master d26f0f7 06other 10SHARD4/pubkeys registration of chosenken on SHARD4 |
|
01:23
🔗
|
SketchCow |
So my question is if chosenken is putting three copies of data in the same physical location. |
|
01:30
🔗
|
iabak-reg |
03registrar 05master 3709c86 06other 10SHARD5/pubkeys registration of chosenken on SHARD5 |
|
01:32
🔗
|
|
chosenken has joined #internetarchive.bak |
|
01:32
🔗
|
|
Start has joined #internetarchive.bak |
|
01:34
🔗
|
chosenken |
Evening. I'm trying to get IA.BAK running using git-annex, but when it goes to checkout from annex the first time I get a permission denied (publickey) |
|
01:40
🔗
|
pikhq |
SketchCow: Looks like he's got technical issues. |
|
01:40
🔗
|
pikhq |
chosenken: Weird, you look to have registered so you *should* be able to just work. |
|
01:41
🔗
|
chosenken |
I had to make a small change to the checkoutshard script to get it to work for me. the two commands to copy the id_rsa keys in won't work on my NFS mount, had to change cp -a to just cp. |
|
01:41
🔗
|
chosenken |
Could that be causing the issue since the files don't have the same permissions? |
|
01:41
🔗
|
pikhq |
Yeah, that could do it. |
|
01:42
🔗
|
pikhq |
Those are SSH public keys, and SSH refuses to use world-readable public keys. |
|
01:42
🔗
|
pikhq |
Weird you'd have issues on NFS though. |
|
01:43
🔗
|
chosenken |
its an NFS shared to a ZFS drive, I don't know why the copy command won't copy permissions. |
|
01:43
🔗
|
pikhq |
A quick fix for now would be to chmod them to be 600. |
|
01:44
🔗
|
chosenken |
I made them 644 so everything can read them (in .git/annex) and still no permission. |
|
01:45
🔗
|
pikhq |
That's just it, *they can't be world readable*. |
|
01:45
🔗
|
pikhq |
SSH will reject them unless they're only readable by the owner. |
|
01:45
🔗
|
chosenken |
ahhhh |
|
01:45
🔗
|
chosenken |
neat |
|
01:45
🔗
|
pikhq |
Erm, sorry, s/public/private/ in my lines. XD |
|
01:46
🔗
|
chosenken |
could it be then that they are owned by nobody? |
|
01:46
🔗
|
pikhq |
Unlikely. |
|
01:46
🔗
|
chosenken |
-rw------- 1 nobody nogroup 1679 May 11 21:33 id_rsa -rw-r--r-- 1 nobody nogroup 404 May 11 21:33 id_rsa.pub |
|
01:46
🔗
|
pikhq |
But a quick ls -l should tell ya |
|
01:46
🔗
|
pikhq |
Oh, huh. |
|
01:46
🔗
|
pikhq |
Well there's your problem. |
|
01:46
🔗
|
pikhq |
If you don't have the ability to make files owned by yourself on that NFS share it's not going to work. :) |
|
01:47
🔗
|
chosenken |
let me work on that then |
|
01:47
🔗
|
chosenken |
thanks! |
|
01:47
🔗
|
pikhq |
(I'm not very familiar with the details of configuring NFS for this, but it should be possible to fix that. Google may elucidate.) |
|
02:08
🔗
|
iabak-reg |
03registrar 05master 50da9f2 06other 10SHARD4/pubkeys registration of chosenken on SHARD4 |
|
02:09
🔗
|
chosenken |
Ha, got it working finally! Had to fix some permissions on the data set in freenas to get it wokring |
|
02:14
🔗
|
|
chosenken has quit IRC (Quit: http://chat.efnet.org ) |
|
02:50
🔗
|
|
niyaje5 has quit IRC (Ping timeout: 600 seconds) |
|
03:00
🔗
|
|
niyaje5 has joined #internetarchive.bak |
|
03:16
🔗
|
|
kalleboo has joined #internetarchive.bak |
|
03:36
🔗
|
|
kalleboo has quit IRC (Ping timeout: 258 seconds) |
|
04:10
🔗
|
|
niyaje5 has quit IRC (Ping timeout: 600 seconds) |
|
04:12
🔗
|
|
niyaje5 has joined #internetarchive.bak |
|
04:15
🔗
|
|
zottelbey has joined #internetarchive.bak |
|
04:28
🔗
|
underscor |
how do I tell if some of my repos have been expired? |
|
04:28
🔗
|
underscor |
is there a way to re-checkin that data? |
|
04:41
🔗
|
|
niyaje5 has quit IRC (Nettalk6 - www.ntalk.de) |
|
04:58
🔗
|
underscor |
http://cumulus.vaped.ventures/munin/vaped.ventures/cumulus.vaped.ventures/if_eth0-day.png wheeee iabak downloading |
|
06:19
🔗
|
|
primus104 has joined #internetarchive.bak |
|
06:50
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
06:51
🔗
|
|
garyrh has quit IRC (http://bnc4free.com/) |
|
06:52
🔗
|
|
garyrh has joined #internetarchive.bak |
|
07:19
🔗
|
|
atomotic has joined #internetarchive.bak |
|
07:33
🔗
|
|
primus104 has joined #internetarchive.bak |
|
08:25
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
08:48
🔗
|
|
DopefishJ has joined #internetarchive.bak |
|
08:56
🔗
|
|
DFJustin has quit IRC (Ping timeout: 740 seconds) |
|
08:58
🔗
|
|
dirt has quit IRC (Read error: Operation timed out) |
|
08:58
🔗
|
|
dirt has joined #internetarchive.bak |
|
09:49
🔗
|
|
primus104 has joined #internetarchive.bak |
|
10:07
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
10:07
🔗
|
|
kalleboo has joined #internetarchive.bak |
|
10:30
🔗
|
iabak-reg |
03registrar 05master b6054e2 06other 10SHARD5/pubkeys registration of alex.buie on SHARD5 |
|
10:38
🔗
|
|
kalleboo has quit IRC (Quit: Leaving...) |
|
11:38
🔗
|
|
antomati_ has joined #internetarchive.bak |
|
11:40
🔗
|
|
antomatic has quit IRC (Read error: Operation timed out) |
|
12:35
🔗
|
|
kalleboo has joined #internetarchive.bak |
|
13:01
🔗
|
|
sankin has joined #internetarchive.bak |
|
13:12
🔗
|
|
atomotic has joined #internetarchive.bak |
|
13:23
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
13:26
🔗
|
|
sankin has quit IRC (Leaving.) |
|
13:28
🔗
|
|
kalleboo has quit IRC (Quit: Leaving...) |
|
13:42
🔗
|
|
db48x has joined #internetarchive.bak |
|
13:48
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
13:52
🔗
|
|
sankin has joined #internetarchive.bak |
|
14:18
🔗
|
|
DopefishJ is now known as DFJustin |
|
14:19
🔗
|
|
svchfoo3 sets mode: +o DFJustin |
|
14:41
🔗
|
closure |
underscor: just re-run iabak, it will unexpire you if you were expired |
|
14:47
🔗
|
Senji |
Why isn't 94e8e6af-4baa-4995-8c61-d995bc44d583 showing up in the repo list for shard3? |
|
14:53
🔗
|
closure |
Senji: expired |
|
14:54
🔗
|
Senji |
Err, how do I unexpire it? |
|
14:54
🔗
|
closure |
Senji: just re-run iabak, it will unexpire you if you were expired |
|
14:54
🔗
|
Senji |
I did that earlier |
|
14:56
🔗
|
closure |
yeah, I see you did that 1243 seconds ago? |
|
14:56
🔗
|
closure |
it takes a little while for unexpire to run |
|
14:59
🔗
|
Senji |
closure: 1243 seconds - err that's about 20 minutes? Seems plausible. I'll check back on it later then :) |
|
15:00
🔗
|
closure |
expiry/unexpiry runs daily |
|
15:00
🔗
|
Senji |
"later" should be "tomorrow" then :) |
|
15:01
🔗
|
closure |
hmm, interesting stair-stepping in the graphs lately |
|
15:04
🔗
|
Senji |
Someone's iabak-hourly isn't running properly? |
|
15:04
🔗
|
sep332 |
maybe someone moved data around, and the old copy hasn't expired so it shows up twice? |
|
15:05
🔗
|
sep332 |
ok looking at the leaderboard that's not that likely |
|
15:05
🔗
|
Senji |
I'd guess that no pair of the repo names for shard4 belong to the same person |
|
15:08
🔗
|
db48x |
if you want to avoid being expired, run iabak-cronjob from time to time |
|
15:08
🔗
|
|
Start has quit IRC (Disconnected.) |
|
15:10
🔗
|
closure |
well, the stair-stepping is around an hour granularity actually I think |
|
15:10
🔗
|
closure |
just, someone seems to be blasting through shard4, until they get it all |
|
15:20
🔗
|
db48x |
oops |
|
15:22
🔗
|
db48x |
closure: updated the pull request |
|
15:24
🔗
|
closure |
ah, persistent is excellent |
|
15:26
🔗
|
db48x |
doesn't look hourly to me: |
|
15:26
🔗
|
db48x |
http://iabak.archiveteam.org:8080/render/?width=1094&height=809&_salt=1431444326.09&areaMode=stacked&from=-24hours&lineWidth=&vtitle=%25&yMax=100&yUnitSystem=si&yMin=0&title=SHARD4%20progress%2C%20%25&target=legendValue%28alias%28color%28scale%28divideSeries%28diffSeries%28keepLastValue%28iabak.shardstats.filecount.shard4%29%2CsumSeries%28keepLastValue%28iabak.shardstats.numcopies.0.shard4%29%2CkeepLastValue%28iabak.shard |
|
15:28
🔗
|
Senji |
"No Data" |
|
15:29
🔗
|
db48x |
Senji: did you copy the whole url? |
|
15:29
🔗
|
Senji |
As far as I can see :-) |
|
15:29
🔗
|
Senji |
It's quite a horrible URL |
|
15:30
🔗
|
db48x |
slightly, yes |
|
15:30
🔗
|
db48x |
go to http://iabak.archiveteam.org:8080/ |
|
15:30
🔗
|
db48x |
on the left-hand side, choose User graphs => db48x => shard4 progress, % |
|
15:31
🔗
|
Senji |
Aha |
|
15:31
🔗
|
Senji |
about 5-6 hour intervals then? |
|
15:31
🔗
|
db48x |
looks irregular |
|
15:32
🔗
|
db48x |
there's one at 17:00, then another at about 20:30, then another at midnight, then one at 05:30 |
|
15:33
🔗
|
db48x |
3.5, 3.5, 5.5 |
|
15:34
🔗
|
Senji |
oh, the first two are very small? |
|
15:34
🔗
|
db48x |
much smaller |
|
15:35
🔗
|
db48x |
however |
|
15:35
🔗
|
Senji |
I'm finding the two shades of green there hard to distinguish :) |
|
15:36
🔗
|
Senji |
Which I think is a defect in my monitor |
|
15:36
🔗
|
db48x |
your eyes are broken :) |
|
15:37
🔗
|
db48x |
so, if you graph iabak.shardstats.leaderboard.*.shard4 |
|
15:37
🔗
|
Senji |
No, it's easly to distinguish if I move my head a metre to the left. I think the viewing angle on this work-provided monitors aren't what you wanted |
|
15:37
🔗
|
Senji |
(my eyes *are* broken, but it's the red receptors, not the green ones) |
|
15:37
🔗
|
db48x |
heh |
|
15:38
🔗
|
db48x |
moving your head changes the colors, on certain lower-quality display types |
|
15:38
🔗
|
db48x |
who has repository a7e494e2-341b-48d4-bbb6-eb168710f0e3? |
|
15:39
🔗
|
db48x |
I don't have a clone of shard4, so I can't see the description |
|
15:40
🔗
|
sep332 |
http://iabak.archiveteam.org/stats/ALL.leaderboard ? |
|
15:40
🔗
|
db48x |
closure: bah. I just noticed that I did this backwards; it should be iabak.shardstats.leaderboard.${shard}.${uuid}, since any given uuid can only be a clone of one particular shard :P |
|
15:40
🔗
|
db48x |
sep332: oh, right |
|
15:41
🔗
|
db48x |
1025123521804: a7e494e2-341b-48d4-bbb6-eb168710f0e3 -- abuie@Debian-78-wheezy-64-minimal:/1/IA.BAK/shard4 |
|
15:44
🔗
|
|
sunny256_ has quit IRC (Read error: Connection reset by peer) |
|
15:48
🔗
|
db48x |
hmm |
|
15:50
🔗
|
db48x |
closure: I'm removing the reminder message; I'm thinking I should add a call to install-fsck-service there to catch upgraders as well: https://github.com/db48x/IA.BAK/commit/0add1e764a6769b4581fb6b7653095c19cc9d4b5 |
|
15:51
🔗
|
|
primus104 has joined #internetarchive.bak |
|
15:52
🔗
|
closure |
keep the reminder; it'll fire if the cron job is broken for some reason |
|
15:53
🔗
|
db48x |
it'll also fire if the user is using systemd |
|
15:53
🔗
|
db48x |
the systemd job puts the log in the journal |
|
15:53
🔗
|
closure |
oic |
|
15:55
🔗
|
|
DFJustin has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
SketchCow has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
csssuf has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
chfoo- has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:55
🔗
|
|
underscor has quit IRC (ircd.shaw.ca irc.shaw.ca) |
|
15:58
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
16:03
🔗
|
|
DFJustin has joined #internetarchive.bak |
|
16:03
🔗
|
|
wp494 has joined #internetarchive.bak |
|
16:03
🔗
|
|
csssuf has joined #internetarchive.bak |
|
16:03
🔗
|
|
chfoo- has joined #internetarchive.bak |
|
16:03
🔗
|
|
matthusby has joined #internetarchive.bak |
|
16:03
🔗
|
|
underscor has joined #internetarchive.bak |
|
16:03
🔗
|
|
irc.shaw.ca sets mode: +o DFJustin |
|
16:13
🔗
|
|
Start has joined #internetarchive.bak |
|
16:45
🔗
|
|
Start has quit IRC (Disconnected.) |
|
16:50
🔗
|
|
Start has joined #internetarchive.bak |
|
17:01
🔗
|
|
SketchCow has joined #internetarchive.bak |
|
17:01
🔗
|
|
svchfoo2 sets mode: +o SketchCow |
|
17:42
🔗
|
|
Start has quit IRC (Disconnected.) |
|
18:44
🔗
|
|
Start has joined #internetarchive.bak |
|
18:44
🔗
|
|
Start has quit IRC (Client Quit) |
|
18:45
🔗
|
|
Start has joined #internetarchive.bak |
|
18:52
🔗
|
|
Start has quit IRC (Ping timeout: 370 seconds) |
|
19:19
🔗
|
closure |
I forget who it was that noticed that annex.diskreserve was not checked right when performing parallel downloads, anyway, that's fixed in git and will be in the next autbuild, if you need it |
|
19:41
🔗
|
|
primus104 has joined #internetarchive.bak |
|
20:53
🔗
|
|
sankin has quit IRC (Leaving.) |
|
21:19
🔗
|
SketchCow |
Our reporting mechanism seems a little odd. |
|
22:21
🔗
|
|
marvinw has quit IRC (Read error: Operation timed out) |
|
22:23
🔗
|
|
sep332 has quit IRC (Read error: Operation timed out) |
|
22:23
🔗
|
|
S[h]O[r]T has quit IRC (Read error: Operation timed out) |
|
22:24
🔗
|
|
Ctrl-S has quit IRC (Read error: Operation timed out) |
|
22:24
🔗
|
|
S[h]O[r]T has joined #internetarchive.bak |
|
22:24
🔗
|
|
Control-S has joined #internetarchive.bak |
|
22:25
🔗
|
|
sep332 has joined #internetarchive.bak |
|
22:26
🔗
|
|
svchfoo2 sets mode: +o sep332 |
|
22:26
🔗
|
|
marvinw has joined #internetarchive.bak |
|
22:42
🔗
|
|
Control-S has quit IRC (Read error: Connection reset by peer) |
|
22:42
🔗
|
|
achip has quit IRC (Read error: Operation timed out) |
|
22:43
🔗
|
|
achip has joined #internetarchive.bak |
|
22:43
🔗
|
|
Control-S has joined #internetarchive.bak |
|
22:47
🔗
|
|
Control-S has quit IRC (Read error: Connection reset by peer) |
|
22:49
🔗
|
|
dirt has quit IRC (Ping timeout: 600 seconds) |
|
22:50
🔗
|
|
dirt has joined #internetarchive.bak |
|
22:53
🔗
|
|
sep332 has quit IRC (Ping timeout: 600 seconds) |
|
22:55
🔗
|
|
S[h]O[r]T has quit IRC (Ping timeout: 600 seconds) |
|
22:56
🔗
|
|
Control-S has joined #internetarchive.bak |
|
22:56
🔗
|
|
S[h]O[r]T has joined #internetarchive.bak |
|
22:58
🔗
|
db48x |
SketchCow: how so? |
|
22:59
🔗
|
|
sep332 has joined #internetarchive.bak |
|
22:59
🔗
|
|
svchfoo3 sets mode: +o sep332 |
|
23:00
🔗
|
SketchCow |
I think there's a huge lag from people downloading stuff and then reporting it as downloaded |
|
23:00
🔗
|
db48x |
yes, quite likely |
|
23:00
🔗
|
db48x |
either they're not syncing often enough, or there's too much contention during syncs |
|
23:00
🔗
|
db48x |
we aren't measuring either |
|
23:04
🔗
|
|
Start has joined #internetarchive.bak |
|
23:09
🔗
|
Senji |
I don't notice many failed syncs |
|
23:09
🔗
|
Senji |
Usually when I do it's because I've stupidly started two syncs at once |
|
23:17
🔗
|
|
kalleboo has joined #internetarchive.bak |