#internetarchive.bak 2015-09-15,Tue

↑back Search

Time Nickname Message
00:08 🔗 Start has joined #internetarchive.bak
00:20 🔗 primus104 has quit IRC (Leaving.)
00:21 🔗 Senji has quit IRC (Read error: Operation timed out)
00:37 🔗 Senji has joined #internetarchive.bak
00:47 🔗 SketchCow I'm seeing repair!
00:48 🔗 SketchCow closure: Should we be recruiting more storage?
01:56 🔗 closure I need to add some shards, need more good collections to put in them
03:52 🔗 SketchCow has quit IRC (Ping timeout: 240 seconds)
04:37 🔗 Senji has quit IRC (Read error: Operation timed out)
04:41 🔗 Senji has joined #internetarchive.bak
04:58 🔗 frontward is now known as Erkan
06:35 🔗 primus104 has joined #internetarchive.bak
07:08 🔗 SketchCow has joined #internetarchive.bak
08:02 🔗 Start has quit IRC (Read error: Connection reset by peer)
08:03 🔗 Start has joined #internetarchive.bak
08:46 🔗 primus104 has quit IRC (Leaving.)
09:46 🔗 atomotic has joined #internetarchive.bak
09:57 🔗 primus104 has joined #internetarchive.bak
10:27 🔗 zhongfu has quit IRC (Quit: Goodbye.)
10:28 🔗 zhongfu has joined #internetarchive.bak
10:44 🔗 atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…)
11:04 🔗 primus104 has quit IRC (Leaving.)
11:05 🔗 Senji has quit IRC (Read error: Operation timed out)
11:16 🔗 Senji has joined #internetarchive.bak
11:27 🔗 db48x has quit IRC (Read error: Connection reset by peer)
11:28 🔗 Senji has quit IRC (Ping timeout: 252 seconds)
11:30 🔗 atomotic has joined #internetarchive.bak
11:35 🔗 Senji has joined #internetarchive.bak
11:37 🔗 db48x` has joined #internetarchive.bak
11:39 🔗 db48x has joined #internetarchive.bak
11:41 🔗 db48x` has quit IRC (Client Quit)
11:51 🔗 Senji has quit IRC (Ping timeout: 252 seconds)
11:55 🔗 primus104 has joined #internetarchive.bak
12:09 🔗 Senji has joined #internetarchive.bak
12:22 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
12:31 🔗 Senji has quit IRC (Ping timeout: 506 seconds)
12:42 🔗 cloudmons has quit IRC (Ping timeout: 492 seconds)
12:51 🔗 Senji has joined #internetarchive.bak
13:07 🔗 atomotic has joined #internetarchive.bak
13:36 🔗 cloudmons has joined #internetarchive.bak
13:43 🔗 primus104 has quit IRC (Leaving.)
13:51 🔗 Start has quit IRC (Quit: Disconnected.)
14:22 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
14:38 🔗 Start has joined #internetarchive.bak
15:15 🔗 primus104 has joined #internetarchive.bak
15:40 🔗 primus104 has quit IRC (Leaving.)
15:51 🔗 Ash___ is now known as Ash
15:51 🔗 higgins_ has joined #internetarchive.bak
15:52 🔗 higgins_ is now known as chrishigg
15:57 🔗 chrishigg Hey, folks. So my first 5TB disk is full (yay!). When this occurs, do I simply add a second disk and start a second instance of ia.bak running? (For context, I have lots of disk lying around, but typically in 2-5TB chunks.)
16:01 🔗 db48x yep
16:03 🔗 Start has quit IRC (Quit: Disconnected.)
16:04 🔗 db48x you'll also need to manually adjust the cron job (or systemd unit) so that the shards in the new location are fscked properly
16:05 🔗 db48x and you'll want to ensure that you're either not downloading the same shards in both locations, or that your shards are ignoring files already in the other copy
16:08 🔗 chrishigg makes sense. does the readme (or similar documentation) tell me where I can include/exclude shards? in my first run, I just kinda let it go nuts, didn't touch any config per se.
16:11 🔗 db48x of course not, that would be crazy
16:11 🔗 db48x you can edit the repolist file
16:12 🔗 db48x you can change the mode to anything you like
16:12 🔗 db48x it'll ignore any mode it doesn't recognize
16:12 🔗 Start has joined #internetarchive.bak
16:28 🔗 DFJustin has quit IRC (Remote host closed the connection)
16:28 🔗 DFJustin has joined #internetarchive.bak
16:57 🔗 primus104 has joined #internetarchive.bak
16:58 🔗 chrishigg Oh right, the repolist file. OK. Thank you!
17:01 🔗 SketchCow https://docs.google.com/spreadsheets/d/1kPZbkiFD_SBf8kJV-fYAu6xzTBuVKAAxzld6grnDUSU/edit?usp=sharing
17:01 🔗 SketchCow To help with finding potential collections, here's a list of every collection we've done so far, sorted for easy finding.
17:02 🔗 db48x chrishigg: you're welcome :)
17:15 🔗 Start has quit IRC (Quit: Disconnected.)
17:25 🔗 closure hmm, I dunno if we need to bother users about not downloading the same shard onto their multiple disks
17:26 🔗 closure I mean, it would not be ideal if someone ended up being the only one that had some of the files in a given shard, but otoh it adds a lot of complexity to avoid it, and it's not fully avoidable anyhow
17:27 🔗 closure SketchCow: IIRC db48x had an idea of a SHARD0 that contained a list like that, and other index information
17:32 🔗 closure oh yeah, we already have that collection list, automatically, here http://iabak.archiveteam.org/stats/ALL.collections (and, my shard generaion scripts even check it, yay)
17:35 🔗 SketchCow Good.
17:55 🔗 beardicus has quit IRC (Quit: bye now)
18:00 🔗 beardicus has joined #internetarchive.bak
19:49 🔗 Start has joined #internetarchive.bak
19:56 🔗 Start has quit IRC (Quit: Disconnected.)
20:11 🔗 Start has joined #internetarchive.bak
20:20 🔗 Start has quit IRC (Quit: Disconnected.)
21:50 🔗 zz_CyberJ is now known as CyberJaco
22:00 🔗 garyrh has quit IRC (Quit: http://bnc4free.com/)
22:09 🔗 chrishigg has quit IRC (hub.se efnet.port80.se)
22:09 🔗 zhongfu has quit IRC (hub.se efnet.port80.se)
22:09 🔗 pikhq has quit IRC (hub.se efnet.port80.se)
22:09 🔗 lhobas has quit IRC (hub.se efnet.port80.se)
22:09 🔗 GLaDOS has quit IRC (hub.se efnet.port80.se)
22:09 🔗 jbenet__ has quit IRC (hub.se efnet.port80.se)
22:09 🔗 Ctrl-S has quit IRC (hub.se efnet.port80.se)
22:09 🔗 gamingrob has quit IRC (hub.se efnet.port80.se)
22:09 🔗 bpye has quit IRC (hub.se efnet.port80.se)
22:09 🔗 ppiixx has quit IRC (hub.se efnet.port80.se)
22:09 🔗 mattl has quit IRC (hub.se efnet.port80.se)
22:09 🔗 Muad-Dib has quit IRC (hub.se efnet.port80.se)
23:02 🔗 Start has joined #internetarchive.bak

irclogger-viewer