| Time |
Nickname |
Message |
|
00:05
🔗
|
|
sevs has quit IRC (Ping timeout: 270 seconds) |
|
01:03
🔗
|
thelsdj |
fusl: if you can at least leave the cronjob then what you have downloaded can be checked and counted, i'm sure db48x will have at least some good questions for you shortly to track down the issue |
|
01:13
🔗
|
iabak-reg |
03registrar 05master ff366a3 06other 10SHARD5/pubkeys registration of fusl on SHARD5 |
|
02:26
🔗
|
|
LightBulb has joined #internetarchive.bak |
|
02:52
🔗
|
|
antomati_ has joined #internetarchive.bak |
|
02:54
🔗
|
|
antomatic has quit IRC (Ping timeout: 260 seconds) |
|
03:09
🔗
|
closure |
db48x: $work wanted a unified git-annex batch command, but I (mostly) convinced them not to pay me to spend all the time that would take for redundant parsing of stuff. |
|
03:10
🔗
|
closure |
Instead, mutiple processes can be run accepting input on stdin and doing different things with it. Concurrently if necessary. If you need fromkey to accept input on stdin, I can add that fairly easily. |
|
04:42
🔗
|
iabak-reg |
03registrar 05master 867bf6f 06other 10SHARD15/pubkeys registration of me on SHARD15 |
|
04:44
🔗
|
iabak-reg |
03registrar 05master 2214a3c 06other 10SHARD14/pubkeys registration of me on SHARD14 |
|
04:46
🔗
|
iabak-reg |
03registrar 05master 85a7250 06other 10SHARD17/pubkeys registration of me on SHARD17 |
|
04:47
🔗
|
iabak-reg |
03registrar 05master 76d982e 06other 10SHARD6/pubkeys registration of me on SHARD6 |
|
04:59
🔗
|
iabak-reg |
03registrar 05master b46147f 06other 10SHARD10/pubkeys registration of me on SHARD10 |
|
05:01
🔗
|
SketchCow |
Rachel Maddow just pushed the Internet Archive heavy |
|
05:01
🔗
|
SketchCow |
Likely will lead to more people joining this |
|
05:09
🔗
|
iabak-reg |
03registrar 05master 89e6b03 06other 10SHARD18/pubkeys registration of me on SHARD18 |
|
05:09
🔗
|
iabak-reg |
03registrar 05master 556474e 06other 10SHARD16/pubkeys registration of me on SHARD16 |
|
05:10
🔗
|
iabak-reg |
03registrar 05master b3c7a2e 06other 10SHARD12/pubkeys registration of me on SHARD12 |
|
05:18
🔗
|
|
Start has joined #internetarchive.bak |
|
05:19
🔗
|
iabak-reg |
03registrar 05master af41e74 06other 10SHARD11/pubkeys registration of me on SHARD11 |
|
05:27
🔗
|
iabak-reg |
03registrar 05master 119acb2 06other 10SHARD19/pubkeys registration of me on SHARD19 |
|
05:27
🔗
|
iabak-reg |
03registrar 05master e8293b8 06other 10SHARD5/pubkeys registration of me on SHARD5 |
|
05:47
🔗
|
|
GrumpyBea has joined #internetarchive.bak |
|
05:52
🔗
|
|
GrumpyBea has quit IRC (Client Quit) |
|
06:10
🔗
|
|
Frogging has quit IRC (hub.efnet.us irc.colosolutions.net) |
|
06:10
🔗
|
|
wp494 has quit IRC (hub.efnet.us irc.colosolutions.net) |
|
06:10
🔗
|
|
balrog has quit IRC (hub.efnet.us irc.colosolutions.net) |
|
06:10
🔗
|
|
joepie91 has quit IRC (hub.efnet.us irc.colosolutions.net) |
|
06:10
🔗
|
|
ivan has quit IRC (hub.efnet.us irc.colosolutions.net) |
|
06:10
🔗
|
|
kyan has quit IRC (Quit: Leaving) |
|
06:20
🔗
|
|
Frogging has joined #internetarchive.bak |
|
06:20
🔗
|
|
wp494 has joined #internetarchive.bak |
|
06:20
🔗
|
|
balrog has joined #internetarchive.bak |
|
06:20
🔗
|
|
joepie91 has joined #internetarchive.bak |
|
06:20
🔗
|
|
ivan has joined #internetarchive.bak |
|
06:20
🔗
|
|
irc.colosolutions.net sets mode: +o balrog |
|
06:29
🔗
|
iabak-reg |
03registrar 05master 4ef9090 06other 10SHARD15/pubkeys registration of grumpybear4257 on SHARD15 |
|
06:41
🔗
|
iabak-reg |
03registrar 05master 160b597 06other 10SHARD16/pubkeys registration of grumpybear4257 on SHARD16 |
|
06:47
🔗
|
iabak-reg |
03registrar 05master ed9e249 06other 10SHARD11/pubkeys registration of grumpybear4257 on SHARD11 |
|
06:54
🔗
|
iabak-reg |
03registrar 05master bec59f6 06other 10SHARD17/pubkeys registration of grumpybear4257 on SHARD17 |
|
07:23
🔗
|
iabak-reg |
03registrar 05master af8f13f 06other 10SHARD18/pubkeys registration of grumpybear4257 on SHARD18 |
|
07:28
🔗
|
|
balrog has quit IRC (Read error: Operation timed out) |
|
07:36
🔗
|
|
balrog has joined #internetarchive.bak |
|
07:37
🔗
|
|
svchfoo3 sets mode: +o balrog |
|
07:42
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
08:22
🔗
|
|
LightBulb has quit IRC (Ping timeout: 268 seconds) |
|
08:22
🔗
|
|
atomotic has joined #internetarchive.bak |
|
08:54
🔗
|
|
coprophag has joined #internetarchive.bak |
|
09:11
🔗
|
|
LightBulb has joined #internetarchive.bak |
|
09:13
🔗
|
LightBulb |
I'm not sure whether this was answered, but I'm getting the message that we've run out of shards to download before I ran out of disk space... |
|
09:25
🔗
|
|
LightBulb has quit IRC (Ping timeout: 268 seconds) |
|
09:50
🔗
|
iabak-reg |
03registrar 05master b0be79d 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
09:50
🔗
|
iabak-reg |
03registrar 05master b85817f 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
09:51
🔗
|
iabak-reg |
03registrar 05master 9da3f85 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
09:51
🔗
|
iabak-reg |
03registrar 05master 8a3b8d4 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
09:51
🔗
|
iabak-reg |
03registrar 05master 8a3b8d4 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
09:52
🔗
|
iabak-reg |
03registrar 05master f133849 06other 10SHARD10/pubkeys registration of mail on SHARD10 |
|
10:32
🔗
|
|
owl has joined #internetarchive.bak |
|
10:40
🔗
|
iabak-reg |
03registrar 05master d272ca0 06other 10SHARD19/pubkeys registration of iabak on SHARD19 |
|
11:07
🔗
|
|
atomotic has quit IRC (Read error: Connection reset by peer) |
|
11:21
🔗
|
|
atomotic has joined #internetarchive.bak |
|
11:44
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
12:43
🔗
|
|
atomotic has joined #internetarchive.bak |
|
13:59
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
14:04
🔗
|
|
atomotic has joined #internetarchive.bak |
|
14:08
🔗
|
|
VADemon has joined #internetarchive.bak |
|
14:37
🔗
|
|
AsmoB has joined #internetarchive.bak |
|
14:43
🔗
|
|
coprophag has quit IRC (Ping timeout: 260 seconds) |
|
14:46
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
15:02
🔗
|
|
Start has joined #internetarchive.bak |
|
15:30
🔗
|
iabak-reg |
03registrar 05master 380eda7 06other 10SHARD17/pubkeys registration of rcc.notify on SHARD17 |
|
15:32
🔗
|
Aoede |
With all the new people with their terabytes coming in, is there a point to keep my 80gb? |
|
15:33
🔗
|
HCross |
YES |
|
15:33
🔗
|
|
coprophag has joined #internetarchive.bak |
|
15:33
🔗
|
|
coprophag has quit IRC (Connection closed) |
|
15:34
🔗
|
Aoede |
Okay, thanks. I would give it more space if I had any :p |
|
15:40
🔗
|
|
atomotic has joined #internetarchive.bak |
|
15:47
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
16:10
🔗
|
|
coprophag has joined #internetarchive.bak |
|
16:11
🔗
|
|
sevs has joined #internetarchive.bak |
|
16:25
🔗
|
|
atomotic has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…) |
|
16:56
🔗
|
|
komarEX has joined #internetarchive.bak |
|
16:56
🔗
|
komarEX |
hey |
|
16:57
🔗
|
komarEX |
I have a bit of problem with download of whole shard4 |
|
16:57
🔗
|
komarEX |
I'm about 2/3 in and iabak say that it's done |
|
16:58
🔗
|
komarEX |
http://pastebin.com/raw/vVCcppDv |
|
16:58
🔗
|
komarEX |
I don't get it |
|
16:58
🔗
|
komarEX |
in repolist I marked shard4 as active |
|
17:00
🔗
|
|
kula has quit IRC (Read error: Operation timed out) |
|
17:00
🔗
|
Kaz |
komarEX: please don't edit repolist locally |
|
17:01
🔗
|
komarEX |
Kaz: why? |
|
17:02
🔗
|
Kaz |
git repo being out of sync probably means you're not going to get other updates once the main repo is updated |
|
17:03
🔗
|
|
kula has joined #internetarchive.bak |
|
17:08
🔗
|
komarEX |
Kaz: ok but that won't fix my problem I guesS? |
|
17:09
🔗
|
Kaz |
no, but it'll fix one of the issues you have |
|
17:10
🔗
|
komarEX |
Kaz: I have other issue? what kind? these failed downloads ? |
|
17:11
🔗
|
Kaz |
your repo is out of sync |
|
17:11
🔗
|
|
atomotic has joined #internetarchive.bak |
|
17:11
🔗
|
Kaz |
so even when there's a fix for your other issue, you won't get it |
|
17:30
🔗
|
komarEX |
and btw. how come about 170k files in shard end up as about 7mil objects in git-annex |
|
18:47
🔗
|
thelsdj |
komarEX: iabak script won't get more because it only gets files that there aren't already enough copies out there, so if you want to download an entire shard you need to run git annex get manually |
|
18:48
🔗
|
thelsdj |
see find_insufficient_copies in iabak-helper |
|
18:51
🔗
|
komarEX |
thelsdj: thanks |
|
19:18
🔗
|
db48x |
komarEX: when you see 'verification of content failed' it means that the file has changed on IA since we created the shards |
|
19:19
🔗
|
db48x |
usually you'll see that the file is named *_meta.xml; that's the file that stores the metadata about the item |
|
19:19
🔗
|
db48x |
so if someone goes in and changes that metadata (for updates or corrections or whatever) then it'll fail to verify |
|
19:20
🔗
|
db48x |
I've written a script that we can use to go back through and correct that, finding the new hashes for files that have changed, but it's not quite ready for use yet |
|
19:50
🔗
|
|
AsmoB has quit IRC (Read error: Connection reset by peer) |
|
20:20
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
20:58
🔗
|
|
db48x has quit IRC (Read error: Operation timed out) |
|
20:59
🔗
|
|
db48x has joined #internetarchive.bak |
|
21:18
🔗
|
|
antomati_ is now known as antomatic |
|
22:10
🔗
|
|
komarEX has quit IRC (Quit: Page closed) |
|
22:11
🔗
|
|
kula has quit IRC (Read error: Operation timed out) |
|
22:14
🔗
|
|
kula has joined #internetarchive.bak |
|
23:01
🔗
|
|
bwn has quit IRC (Ping timeout: 244 seconds) |
|
23:09
🔗
|
|
bwn has joined #internetarchive.bak |
|
23:09
🔗
|
|
kyan has joined #internetarchive.bak |
|
23:37
🔗
|
|
db48x has quit IRC (Read error: Operation timed out) |
|
23:38
🔗
|
|
db48x has joined #internetarchive.bak |