| Time |
Nickname |
Message |
|
00:01
🔗
|
thelsdj |
ok i'm removing NOMORE and seeing if when its done with 8 hours on one shard will start another |
|
00:01
🔗
|
db48x |
ah, if you have NOMORE set then yea |
|
00:02
🔗
|
thelsdj |
yeah i forgot about that |
|
00:03
🔗
|
db48x |
currently when you run iabak we first fastfsck every shard you have, then check every shard you already have to see if there are any files that need to be backed up (because someone left, for example) |
|
00:03
🔗
|
db48x |
then if you still have space and you haven't set NOMORE, then we look for new shards to download |
|
00:03
🔗
|
thelsdj |
so i guess its probably better that I have a bit from a bunch of shards than all of one, lets it be more helpful |
|
00:05
🔗
|
db48x |
thelsdj: hopefully, yes |
|
00:05
🔗
|
db48x |
haven't actually proved it one way or the other, yet |
|
00:07
🔗
|
|
sevs has quit IRC (Ping timeout: 268 seconds) |
|
00:34
🔗
|
iabak-reg |
03registrar 05master 8530e34 06other 10SHARD18/pubkeys registration of hcross on SHARD18 |
|
01:14
🔗
|
|
bwn has quit IRC (Ping timeout: 244 seconds) |
|
01:22
🔗
|
|
bwn has joined #internetarchive.bak |
|
01:36
🔗
|
iabak-reg |
03registrar 05master 22fa560 06other 10SHARD9/pubkeys registration of mitch on SHARD9 |
|
02:19
🔗
|
|
VADemon has quit IRC (left4dead) |
|
03:15
🔗
|
|
fusl has quit IRC (Read error: Operation timed out) |
|
03:16
🔗
|
|
fusl has joined #internetarchive.bak |
|
03:45
🔗
|
iabak-reg |
03registrar 05master be3bd29 06other 10SHARD14/pubkeys registration of alex on SHARD14 |
|
03:50
🔗
|
SketchCow |
Looking good, db48x |
|
03:53
🔗
|
iabak-reg |
03registrar 05master 9834eba 06other 10SHARD11/pubkeys registration of alex on SHARD11 |
|
04:18
🔗
|
iabak-reg |
03registrar 05master dcc71ee 06other 10SHARD4/pubkeys registration of thelsdj on SHARD4 |
|
04:19
🔗
|
db48x |
it is good to see new registrations |
|
04:20
🔗
|
SketchCow |
Yeah, I am glad we're fixing all these little minor things |
|
04:20
🔗
|
SketchCow |
and the major things |
|
04:28
🔗
|
|
db48x` has joined #internetarchive.bak |
|
04:30
🔗
|
|
db48x has quit IRC (Read error: Operation timed out) |
|
04:31
🔗
|
|
db48x` is now known as db48x |
|
05:22
🔗
|
|
kyan has quit IRC (Quit: Leaving) |
|
06:57
🔗
|
iabak-reg |
03registrar 05master 26f3de2 06other 10SHARD3/pubkeys registration of alex on SHARD3 |
|
07:24
🔗
|
|
yipdw has quit IRC (Remote host closed the connection) |
|
07:46
🔗
|
|
yipdw has joined #internetarchive.bak |
|
07:46
🔗
|
|
svchfoo3 sets mode: +o yipdw |
|
07:50
🔗
|
iabak-reg |
03registrar 05master 61398d4 06other 10SHARD16/pubkeys registration of milenko on SHARD16 |
|
08:06
🔗
|
iabak-reg |
03registrar 05master b1e3072 06other 10SHARD4/pubkeys registration of deewiant+ia.bak on SHARD4 |
|
08:08
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
08:09
🔗
|
|
Start_ has joined #internetarchive.bak |
|
08:12
🔗
|
iabak-reg |
03registrar 05master 411ab84 06other 10SHARD3/pubkeys registration of milenko on SHARD3 |
|
08:40
🔗
|
|
Start_ has quit IRC (Quit: Disconnected.) |
|
08:43
🔗
|
|
sevs has joined #internetarchive.bak |
|
08:45
🔗
|
iabak-reg |
03registrar 05master cf223cb 06other 10SHARD14/pubkeys registration of sevs44936-iabak on SHARD14 |
|
08:49
🔗
|
iabak-reg |
03registrar 05master 6e096ca 06other 10SHARD15/pubkeys registration of sevs44936-iabak on SHARD15 |
|
08:50
🔗
|
iabak-reg |
03registrar 05master 3738c02 06other 10SHARD12/pubkeys registration of sevs44936-iabak on SHARD12 |
|
08:53
🔗
|
sevs |
seems there is something wrong there |
|
08:55
🔗
|
sevs |
iabak switches to a new shard, creates the directory, starts the shard and finds no items to download - switches to another shard |
|
08:56
🔗
|
db48x |
sevs: shards 14, 15, and 12 are not so complete that it should find no items to download |
|
08:57
🔗
|
sevs |
http://pastebin.com/1gAyxs6q |
|
08:57
🔗
|
sevs |
db48x: yeah, obviously |
|
08:58
🔗
|
db48x |
try running it with -x? |
|
08:58
🔗
|
db48x |
also, what's up with the error message from flock? |
|
08:59
🔗
|
iabak-reg |
03registrar 05master ade3968 06other 10SHARD10/pubkeys registration of sevs44936-iabak on SHARD10 |
|
08:59
🔗
|
sevs |
./iabak -x ? |
|
09:01
🔗
|
db48x |
sorry, no. edit iabak-helper to include set -x at the top, then run iabak as normal |
|
09:01
🔗
|
sevs |
re flock always had that message, no idea whats up with that |
|
09:02
🔗
|
iabak-reg |
03registrar 05master 0c78de6 06other 10SHARD16/pubkeys registration of sevs44936-iabak on SHARD16 |
|
09:03
🔗
|
db48x |
ok. it's problematic but probably not fatal |
|
09:07
🔗
|
sevs |
http://pastebin.com/Rj5truxN |
|
09:08
🔗
|
db48x |
oh, nice |
|
09:08
🔗
|
db48x |
line 179 |
|
09:09
🔗
|
db48x |
I wasn't aware that we had any files that large in any of the active shards |
|
09:09
🔗
|
sevs |
i've seen 120g+ |
|
09:10
🔗
|
db48x |
what are people thinking? |
|
09:10
🔗
|
db48x |
ok, edit your copy |
|
09:10
🔗
|
yipdw |
probably the same thing we thought with the 50 GB megawarcs in the early AT packs |
|
09:11
🔗
|
yipdw |
except on a worse scale |
|
09:11
🔗
|
db48x |
on lines 166 and 167 of iabak-helper, change $((8*4*1024*1024*1024)) to $((120*1024*1024*1024)) |
|
09:12
🔗
|
db48x |
I will change it so that it always downloads at least one file :P |
|
09:12
🔗
|
sevs |
ok, running |
|
09:12
🔗
|
db48x |
file or item |
|
09:13
🔗
|
sevs |
nope |
|
09:13
🔗
|
sevs |
same as before |
|
09:13
🔗
|
db48x |
what was the file size it objected to? |
|
09:14
🔗
|
sevs |
+ [[ 212044756224 -lt 128849018880 ]] |
|
09:14
🔗
|
sevs |
212G ? |
|
09:14
🔗
|
db48x |
lol |
|
09:15
🔗
|
db48x |
ok, here's another option |
|
09:15
🔗
|
db48x |
line 181: while read -d $'\0' bytes filename && [[ ${numfiles} -eq 0 || $((${spaceneeded} + ${bytes})) -lt ${spacelimit} ]]; do |
|
09:16
🔗
|
sevs |
should I revert the other change? |
|
09:16
🔗
|
db48x |
if you want |
|
09:17
🔗
|
db48x |
the exact size of that threshold is up for debate |
|
09:18
🔗
|
sevs |
YES, success |
|
09:18
🔗
|
db48x |
thanks for the bug report :D |
|
09:18
🔗
|
sevs |
np, :D |
|
10:20
🔗
|
iabak-reg |
03registrar 05master 21342e8 06other 10SHARD16/pubkeys registration of sevs44936-iabak on SHARD16 |
|
10:21
🔗
|
iabak-reg |
03registrar 05master 5cbe98a 06other 10SHARD10/pubkeys registration of sevs44936-iabak on SHARD10 |
|
10:34
🔗
|
|
wp494 has quit IRC (Ping timeout: 506 seconds) |
|
10:38
🔗
|
|
wp494 has joined #internetarchive.bak |
|
10:40
🔗
|
iabak-reg |
03registrar 05master e6ff507 06other 10SHARD11/pubkeys registration of meroje on SHARD11 |
|
10:40
🔗
|
|
asktoomuc has joined #internetarchive.bak |
|
10:42
🔗
|
asktoomuc |
I have a question. Since I've been having issues with the iabak script for the past few days, I have relaunched it multiple times and pretty much everytime it started downloading a new shard. Now I have at least 5 of them in my local repo. Is it a problem? Should I cleanup or start over with the latest version of the script? |
|
11:14
🔗
|
db48x |
asktoomuc: no, you're good |
|
11:16
🔗
|
|
bwn has quit IRC (Ping timeout: 961 seconds) |
|
11:22
🔗
|
asktoomuc |
ok, I'll take your word for it :) |
|
12:06
🔗
|
iabak-reg |
03registrar 05master df6cf89 06other 10SHARD14/pubkeys registration of thelsdj on SHARD14 |
|
12:30
🔗
|
db48x |
asktoomuc: having multiple shards is only a problem if you value the files in one particular shard more than the ones in the other shards |
|
12:30
🔗
|
asktoomuc |
yeah, not really. I just want to help with the backup |
|
12:39
🔗
|
|
db48x has quit IRC (Read error: Operation timed out) |
|
12:40
🔗
|
|
db48x has joined #internetarchive.bak |
|
13:02
🔗
|
|
bwn has joined #internetarchive.bak |
|
13:37
🔗
|
|
atomotic has joined #internetarchive.bak |
|
14:54
🔗
|
sep332 |
db48x: do you remember what you wanted me for like... 2 days ago |
|
15:06
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
15:44
🔗
|
|
kyan has joined #internetarchive.bak |
|
15:52
🔗
|
iabak-reg |
03registrar 05master 1597a88 06other 10SHARD4/pubkeys registration of milenko on SHARD4 |
|
15:58
🔗
|
iabak-reg |
03registrar 05master 8bb712f 06other 10SHARD12/pubkeys registration of alex on SHARD12 |
|
16:15
🔗
|
|
sevs has quit IRC (Ping timeout: 268 seconds) |
|
16:36
🔗
|
|
kyan has quit IRC (Remote host closed the connection) |
|
16:45
🔗
|
|
bwn has quit IRC (Ping timeout: 244 seconds) |
|
16:46
🔗
|
|
atomotic has joined #internetarchive.bak |
|
17:11
🔗
|
|
bwn has joined #internetarchive.bak |
|
17:17
🔗
|
|
kyan has joined #internetarchive.bak |
|
17:31
🔗
|
|
kyan has quit IRC (Read error: Operation timed out) |
|
18:12
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
18:15
🔗
|
iabak-reg |
03registrar 05master b22a87e 06other 10SHARD16/pubkeys registration of milenko on SHARD16 |
|
18:19
🔗
|
thelsdj |
is there a problem with shard4? i've gotten this repeatedly even if i delete the directory and run iabak again, when it tries to get shard4 i get this error: https://gist.github.com/adamipc/34f14c115cfc66cfb3c94133c61f39a0 |
|
18:46
🔗
|
milenko |
I've gotten that error on other shards today... |
|
18:47
🔗
|
milenko |
restarted client and it went past that error and started dl'ing |
|
18:47
🔗
|
milenko |
so idk |
|
18:49
🔗
|
|
asktoomuc has quit IRC (Bye) |
|
18:50
🔗
|
|
asktoomuc has joined #internetarchive.bak |
|
18:50
🔗
|
asktoomuc |
" If you're using NFS then you'll have to install git-annex manually (as the default install tarball uses symlinks), and you'll have to add "-c annex.sshcaching=false" to the ANNEXGETOPTS file so that git-annex doesn't try to create unix sockets on your NFS filesystem." |
|
18:50
🔗
|
asktoomuc |
I didn't have that |
|
18:50
🔗
|
asktoomuc |
is it a new line below the -J setting ? |
|
18:50
🔗
|
asktoomuc |
or on the same line? |
|
18:53
🔗
|
thelsdj |
i think same line, separated by a space |
|
18:53
🔗
|
thelsdj |
-J9 -c .... |
|
18:53
🔗
|
asktoomuc |
I assumed the same, yes |
|
18:53
🔗
|
asktoomuc |
thanks! |
|
19:02
🔗
|
|
atomotic has joined #internetarchive.bak |
|
19:21
🔗
|
iabak-reg |
03registrar 05master 71420a5 06other 10SHARD4/pubkeys registration of alex on SHARD4 |
|
19:30
🔗
|
asktoomuc |
hmmm, not very important, but the contributor page (http://iabak.archiveteam.org/SHARD4.html) is using the email address as a handle. Any way to change that? (or change the email) |
|
19:35
🔗
|
|
Start has joined #internetarchive.bak |
|
19:37
🔗
|
|
kyan has joined #internetarchive.bak |
|
20:03
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
20:08
🔗
|
|
Start has joined #internetarchive.bak |
|
20:10
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
20:28
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
20:42
🔗
|
asktoomuc |
I see a few of these: "git-annex: unable to decommit memory: Invalid argument" |
|
20:49
🔗
|
sep332 |
$ git config user.email "foo@example.com" |
|
20:49
🔗
|
sep332 |
If you've already started a shard you might have to do it again inside each shard folder you already have |
|
21:02
🔗
|
asktoomuc |
understood, thanks :) |
|
21:07
🔗
|
asktoomuc |
hmmm ok that's not it. I see user.email=root@iabak which is what is in the raw data of the statistic page, but is not the handle used on this chan when you register a new shard or the one showing in the summary on the main page |
|
21:34
🔗
|
|
qwebirc13 has joined #internetarchive.bak |
|
21:35
🔗
|
|
qwebirc13 has quit IRC (Client Quit) |
|
21:55
🔗
|
Frogging |
wget seems to have hung, on a recvfrom(6, |
|
21:55
🔗
|
Frogging |
where 6 is this socket: oblivion:47718->ia802305.us.archive.org:https (ESTABLISHED) |
|
21:57
🔗
|
Frogging |
output file's mtime is almost 15 hours ago |
|
21:57
🔗
|
Frogging |
why hasn't it timed out... |
|
22:02
🔗
|
|
asktoomuc has quit IRC (Quit: Page closed) |
|
22:15
🔗
|
iabak-reg |
03registrar 05master a37b504 06other 10SHARD9/pubkeys registration of me on SHARD9 |
|
22:27
🔗
|
iabak-reg |
03registrar 05master d0285b8 06other 10SHARD16/pubkeys registration of me on SHARD16 |
|
22:29
🔗
|
|
sep332_ has joined #internetarchive.bak |
|
22:29
🔗
|
|
sep332 has quit IRC (Quit: konversation out) |
|
22:33
🔗
|
iabak-reg |
03registrar 05master dc9b8e3 06other 10SHARD14/pubkeys registration of me on SHARD14 |
|
22:34
🔗
|
iabak-reg |
03registrar 05master e499e94 06other 10SHARD11/pubkeys registration of me on SHARD11 |
|
22:43
🔗
|
iabak-reg |
03registrar 05master 391abf3 06other 10SHARD15/pubkeys registration of me on SHARD15 |
|
22:53
🔗
|
|
Lord_Nigh has quit IRC (Read error: Operation timed out) |
|
22:53
🔗
|
iabak-reg |
03registrar 05master 80bc028 06other 10SHARD12/pubkeys registration of me on SHARD12 |
|
22:53
🔗
|
iabak-reg |
03registrar 05master 9666db2 06other 10SHARD12/pubkeys registration of me on SHARD12 |
|
23:13
🔗
|
db48x |
sep332_: I do remember :) |
|
23:14
🔗
|
db48x |
sep332_: I saw your name go by on a lot of unavailable files in shard3, and thought we could swap logins |
|
23:35
🔗
|
|
VADemon has joined #internetarchive.bak |
|
23:54
🔗
|
iabak-reg |
03registrar 05master d4ba1e5 06other 10SHARD10/pubkeys registration of me on SHARD10 |