| Time |
Nickname |
Message |
|
00:08
🔗
|
|
Start has joined #internetarchive.bak |
|
00:20
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
00:21
🔗
|
|
Senji has quit IRC (Read error: Operation timed out) |
|
00:37
🔗
|
|
Senji has joined #internetarchive.bak |
|
00:47
🔗
|
SketchCow |
I'm seeing repair! |
|
00:48
🔗
|
SketchCow |
closure: Should we be recruiting more storage? |
|
01:56
🔗
|
closure |
I need to add some shards, need more good collections to put in them |
|
03:52
🔗
|
|
SketchCow has quit IRC (Ping timeout: 240 seconds) |
|
04:37
🔗
|
|
Senji has quit IRC (Read error: Operation timed out) |
|
04:41
🔗
|
|
Senji has joined #internetarchive.bak |
|
04:58
🔗
|
|
frontward is now known as Erkan |
|
06:35
🔗
|
|
primus104 has joined #internetarchive.bak |
|
07:08
🔗
|
|
SketchCow has joined #internetarchive.bak |
|
08:02
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
08:03
🔗
|
|
Start has joined #internetarchive.bak |
|
08:46
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
09:46
🔗
|
|
atomotic has joined #internetarchive.bak |
|
09:57
🔗
|
|
primus104 has joined #internetarchive.bak |
|
10:27
🔗
|
|
zhongfu has quit IRC (Quit: Goodbye.) |
|
10:28
🔗
|
|
zhongfu has joined #internetarchive.bak |
|
10:44
🔗
|
|
atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) |
|
11:04
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
11:05
🔗
|
|
Senji has quit IRC (Read error: Operation timed out) |
|
11:16
🔗
|
|
Senji has joined #internetarchive.bak |
|
11:27
🔗
|
|
db48x has quit IRC (Read error: Connection reset by peer) |
|
11:28
🔗
|
|
Senji has quit IRC (Ping timeout: 252 seconds) |
|
11:30
🔗
|
|
atomotic has joined #internetarchive.bak |
|
11:35
🔗
|
|
Senji has joined #internetarchive.bak |
|
11:37
🔗
|
|
db48x` has joined #internetarchive.bak |
|
11:39
🔗
|
|
db48x has joined #internetarchive.bak |
|
11:41
🔗
|
|
db48x` has quit IRC (Client Quit) |
|
11:51
🔗
|
|
Senji has quit IRC (Ping timeout: 252 seconds) |
|
11:55
🔗
|
|
primus104 has joined #internetarchive.bak |
|
12:09
🔗
|
|
Senji has joined #internetarchive.bak |
|
12:22
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
12:31
🔗
|
|
Senji has quit IRC (Ping timeout: 506 seconds) |
|
12:42
🔗
|
|
cloudmons has quit IRC (Ping timeout: 492 seconds) |
|
12:51
🔗
|
|
Senji has joined #internetarchive.bak |
|
13:07
🔗
|
|
atomotic has joined #internetarchive.bak |
|
13:36
🔗
|
|
cloudmons has joined #internetarchive.bak |
|
13:43
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
13:51
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
14:22
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
14:38
🔗
|
|
Start has joined #internetarchive.bak |
|
15:15
🔗
|
|
primus104 has joined #internetarchive.bak |
|
15:40
🔗
|
|
primus104 has quit IRC (Leaving.) |
|
15:51
🔗
|
|
Ash___ is now known as Ash |
|
15:51
🔗
|
|
higgins_ has joined #internetarchive.bak |
|
15:52
🔗
|
|
higgins_ is now known as chrishigg |
|
15:57
🔗
|
chrishigg |
Hey, folks. So my first 5TB disk is full (yay!). When this occurs, do I simply add a second disk and start a second instance of ia.bak running? (For context, I have lots of disk lying around, but typically in 2-5TB chunks.) |
|
16:01
🔗
|
db48x |
yep |
|
16:03
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
16:04
🔗
|
db48x |
you'll also need to manually adjust the cron job (or systemd unit) so that the shards in the new location are fscked properly |
|
16:05
🔗
|
db48x |
and you'll want to ensure that you're either not downloading the same shards in both locations, or that your shards are ignoring files already in the other copy |
|
16:08
🔗
|
chrishigg |
makes sense. does the readme (or similar documentation) tell me where I can include/exclude shards? in my first run, I just kinda let it go nuts, didn't touch any config per se. |
|
16:11
🔗
|
db48x |
of course not, that would be crazy |
|
16:11
🔗
|
db48x |
you can edit the repolist file |
|
16:12
🔗
|
db48x |
you can change the mode to anything you like |
|
16:12
🔗
|
db48x |
it'll ignore any mode it doesn't recognize |
|
16:12
🔗
|
|
Start has joined #internetarchive.bak |
|
16:28
🔗
|
|
DFJustin has quit IRC (Remote host closed the connection) |
|
16:28
🔗
|
|
DFJustin has joined #internetarchive.bak |
|
16:57
🔗
|
|
primus104 has joined #internetarchive.bak |
|
16:58
🔗
|
chrishigg |
Oh right, the repolist file. OK. Thank you! |
|
17:01
🔗
|
SketchCow |
https://docs.google.com/spreadsheets/d/1kPZbkiFD_SBf8kJV-fYAu6xzTBuVKAAxzld6grnDUSU/edit?usp=sharing |
|
17:01
🔗
|
SketchCow |
To help with finding potential collections, here's a list of every collection we've done so far, sorted for easy finding. |
|
17:02
🔗
|
db48x |
chrishigg: you're welcome :) |
|
17:15
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
17:25
🔗
|
closure |
hmm, I dunno if we need to bother users about not downloading the same shard onto their multiple disks |
|
17:26
🔗
|
closure |
I mean, it would not be ideal if someone ended up being the only one that had some of the files in a given shard, but otoh it adds a lot of complexity to avoid it, and it's not fully avoidable anyhow |
|
17:27
🔗
|
closure |
SketchCow: IIRC db48x had an idea of a SHARD0 that contained a list like that, and other index information |
|
17:32
🔗
|
closure |
oh yeah, we already have that collection list, automatically, here http://iabak.archiveteam.org/stats/ALL.collections (and, my shard generaion scripts even check it, yay) |
|
17:35
🔗
|
SketchCow |
Good. |
|
17:55
🔗
|
|
beardicus has quit IRC (Quit: bye now) |
|
18:00
🔗
|
|
beardicus has joined #internetarchive.bak |
|
19:49
🔗
|
|
Start has joined #internetarchive.bak |
|
19:56
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
20:11
🔗
|
|
Start has joined #internetarchive.bak |
|
20:20
🔗
|
|
Start has quit IRC (Quit: Disconnected.) |
|
21:50
🔗
|
|
zz_CyberJ is now known as CyberJaco |
|
22:00
🔗
|
|
garyrh has quit IRC (Quit: http://bnc4free.com/) |
|
22:09
🔗
|
|
chrishigg has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
zhongfu has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
pikhq has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
lhobas has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
GLaDOS has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
jbenet__ has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
Ctrl-S has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
gamingrob has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
bpye has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
ppiixx has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
mattl has quit IRC (hub.se efnet.port80.se) |
|
22:09
🔗
|
|
Muad-Dib has quit IRC (hub.se efnet.port80.se) |
|
23:02
🔗
|
|
Start has joined #internetarchive.bak |