Time |
Nickname |
Message |
00:07
🔗
|
|
niyaje3 has joined #internetarchive.bak |
00:09
🔗
|
|
niyaje4 has joined #internetarchive.bak |
00:16
🔗
|
|
niyaje3 has quit IRC (Read error: Operation timed out) |
00:18
🔗
|
|
svchfoo2 has joined #internetarchive.bak |
00:19
🔗
|
|
svchfoo1 sets mode: +o svchfoo2 |
00:34
🔗
|
|
svchfoo2 has quit IRC (Quit: Closing) |
00:34
🔗
|
|
svchfoo2 has joined #internetarchive.bak |
00:35
🔗
|
|
svchfoo3 sets mode: +o svchfoo2 |
00:47
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
01:01
🔗
|
|
niyaje4 has joined #internetarchive.bak |
01:05
🔗
|
|
Start has joined #internetarchive.bak |
01:05
🔗
|
|
Start has quit IRC (Client Quit) |
01:06
🔗
|
|
Start has joined #internetarchive.bak |
01:16
🔗
|
|
wp494 has quit IRC (Ping timeout: 740 seconds) |
01:16
🔗
|
|
wp494_ has joined #internetarchive.bak |
01:16
🔗
|
|
wp494_ is now known as wp494 |
01:29
🔗
|
closure |
ppiixx: cute, I'll bet that's the xml index files, that don't have a known size, so it continues downloading them past the diskreserve setting |
01:30
🔗
|
* |
closure fixes.. |
01:30
🔗
|
|
tpw_rules has left Evil will always triumph, because good is dumb. |
01:30
🔗
|
|
tpw_rules has joined #internetarchive.bak |
02:07
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
02:36
🔗
|
|
niyaje4 has joined #internetarchive.bak |
02:50
🔗
|
|
zottelbey has quit IRC (Remote host closed the connection) |
03:32
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
04:09
🔗
|
|
DopefishJ is now known as DFJustin |
04:10
🔗
|
|
svchfoo1 sets mode: +o DFJustin |
04:54
🔗
|
|
SketchCow has quit IRC (Read error: Connection reset by peer) |
04:54
🔗
|
|
chfoo has quit IRC (Ping timeout: 306 seconds) |
04:55
🔗
|
|
Quile_ has joined #internetarchive.bak |
04:56
🔗
|
|
espes___ has quit IRC (ny.us.hub irc.teksavvy.ca) |
04:56
🔗
|
|
Quile has quit IRC (ny.us.hub irc.teksavvy.ca) |
04:56
🔗
|
|
closure has quit IRC (ny.us.hub irc.teksavvy.ca) |
04:56
🔗
|
|
espes___ has joined #internetarchive.bak |
04:56
🔗
|
|
closure has joined #internetarchive.bak |
04:56
🔗
|
|
irc.teksavvy.ca sets mode: +o closure |
04:56
🔗
|
|
chfoo has joined #internetarchive.bak |
04:58
🔗
|
|
espes___ has quit IRC (ircd.choopa.net irc.teksavvy.ca) |
04:58
🔗
|
|
closure has quit IRC (ircd.choopa.net irc.teksavvy.ca) |
05:03
🔗
|
|
espes__ has joined #internetarchive.bak |
05:05
🔗
|
|
svchfoo3 has quit IRC (Quit: Closing) |
05:06
🔗
|
|
svchfoo3 has joined #internetarchive.bak |
05:07
🔗
|
|
svchfoo1 sets mode: +o svchfoo3 |
05:20
🔗
|
|
closure has joined #internetarchive.bak |
05:20
🔗
|
|
svchfoo1 sets mode: +o closure |
06:06
🔗
|
|
raylee has joined #internetarchive.bak |
06:56
🔗
|
|
yipdw_ has joined #internetarchive.bak |
06:58
🔗
|
|
yipdw has quit IRC (Read error: Operation timed out) |
07:15
🔗
|
ppiixx |
closure: yeah exactly that |
07:27
🔗
|
|
fenn has quit IRC (Read error: Operation timed out) |
07:55
🔗
|
|
yipdw_ is now known as yipdw |
07:56
🔗
|
|
svchfoo3 sets mode: +o yipdw |
08:07
🔗
|
|
Infreq has joined #internetarchive.bak |
09:46
🔗
|
|
zottelbey has joined #internetarchive.bak |
14:43
🔗
|
|
VADemon has joined #internetarchive.bak |
15:03
🔗
|
|
atomotic has joined #internetarchive.bak |
16:16
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
16:55
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
16:55
🔗
|
|
Start_ has joined #internetarchive.bak |
19:49
🔗
|
|
SN4T14_ has joined #internetarchive.bak |
19:55
🔗
|
|
SN4T14 has quit IRC (Ping timeout: 369 seconds) |
20:17
🔗
|
|
Start_ is now known as Start |
20:54
🔗
|
|
wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) |
20:58
🔗
|
|
wp494 has joined #internetarchive.bak |
21:04
🔗
|
|
antomati_ is now known as antomatic |
21:31
🔗
|
closure |
tpw_rules: hey, you're at the top of http://iabak.archiveteam.org/stats/SHARD1.expireleaderboard , can you please run iabak periodically? |
21:31
🔗
|
tpw_rules |
what does that mean |
21:31
🔗
|
tpw_rules |
i haven't installed iabak yet |
21:31
🔗
|
closure |
well, you need to |
21:32
🔗
|
tpw_rules |
also i think that may be me from when i erased my first archive |
21:32
🔗
|
tpw_rules |
lemme check my id |
21:32
🔗
|
closure |
I suspect one of them is, and one of them isn't |
21:32
🔗
|
closure |
periodically running iabak is how we'll know which repos still exist |
21:32
🔗
|
tpw_rules |
ok. how do i check my id? and what does iabak do? |
21:33
🔗
|
tpw_rules |
https://archive.org/download/Ttscribe/Ttscribe_files.xml is still darked and preventing me from doing a complete get |
21:34
🔗
|
closure |
I suppose that more files will dark from time to time, I'm not worrying about that |
21:34
🔗
|
tpw_rules |
i'm running fsck |
21:35
🔗
|
closure |
are you running sync after? Are you using the right version of git-annex? iabak takes care of that stuff |
21:35
🔗
|
tpw_rules |
yeah and yeah. i meant this second |
21:37
🔗
|
tpw_rules |
how often do i come up for expiry? i'll set a cron job |
21:39
🔗
|
closure |
also root@katie, root@iashard-de-01, root@iashard-lax-01 |
21:39
🔗
|
closure |
currently we're looking at 1 week, may change |
21:39
🔗
|
tpw_rules |
is fsck --fast enough? |
21:39
🔗
|
closure |
it's too much. iabak has a faster method |
21:40
🔗
|
tpw_rules |
yeah i saw it |
21:40
🔗
|
tpw_rules |
can i just run iabak in a screen and leave it alone? |
21:41
🔗
|
closure |
if you touch IA.BAK/NOMORE, iabak won't check out any more shards at all, and will just do maintenance |
21:42
🔗
|
closure |
it doesn't currently keep running forever though.. can run from cron job |
21:42
🔗
|
tpw_rules |
ok |
21:42
🔗
|
tpw_rules |
is it re-entrant from a cron-job |
21:42
🔗
|
closure |
ym run from cron and from command line? |
21:42
🔗
|
tpw_rules |
i mean like if i run it daily and it starts a download which takes > day |
21:43
🔗
|
tpw_rules |
will it realize it's alrady running and exit |
21:43
🔗
|
closure |
not currently |
21:43
🔗
|
closure |
people like to run multiple ones to use more BW |
21:44
🔗
|
closure |
we could have a iabak-cronjob that is safe that way, and avoids the more expensive stuff |
21:44
🔗
|
closure |
ie, doesn't download more shards |
21:44
🔗
|
tpw_rules |
oh yeah, ok. forgot it dodn't matter |
21:45
🔗
|
tpw_rules |
could you add a 'destination' system to git annex? |
21:45
🔗
|
tpw_rules |
so i can say "put x GB here and y GB there" and it will fill them up as much as possible |
21:46
🔗
|
tpw_rules |
ideally stuffing with little files to get as close to the max as possible |
21:46
🔗
|
tpw_rules |
cause i have a billion hard drives just lying around and i want to attach them to one system |
21:46
🔗
|
closure |
that's accomplished by having different clones of the repo in different places |
21:46
🔗
|
yipdw |
use lvm :P |
21:46
🔗
|
tpw_rules |
i don't want them to duplicate files though |
21:47
🔗
|
tpw_rules |
yipdw: the problem with that is that if one drive fails, everything goes down |
21:47
🔗
|
yipdw |
zfs in raidz2/z3 mode |
21:47
🔗
|
closure |
you can teach git-annex that a repo doesn't want files that are in another repo |
21:48
🔗
|
closure |
or you can move files manually from one to another |
21:48
🔗
|
tpw_rules |
but they're different sizes and that wastes space on redundancy |
21:48
🔗
|
yipdw |
well |
21:48
🔗
|
tpw_rules |
i just want this for the couple dozen laptop and desktop drives i have lying around to be useful (and also stress-test if i need one for something) |
21:48
🔗
|
closure |
cd shard1; git remote add otherdrvie /other/drive/shard1 ; git annex move --to otherdrive |
21:49
🔗
|
yipdw |
it sounds like there's higher-layer solutions available so that's fine, but I'm usually a fan of pushing the physical->logical mapping deeper into the stack so I don't have to care |
21:49
🔗
|
tpw_rules |
yipdw: the problem is the physical map is fairly fluid |
21:50
🔗
|
|
Quile_ has quit IRC (Read error: Operation timed out) |
21:50
🔗
|
|
Quile has joined #internetarchive.bak |
21:50
🔗
|
tpw_rules |
and a drive dying killing the entire thing would be a waste of bandwidth and time |
21:51
🔗
|
tpw_rules |
closure: how often does the leaderboard update? sync is done |
21:53
🔗
|
closure |
only once an hour |
22:07
🔗
|
tpw_rules |
so it turns out i actually have spare hard drives out my butt. found four enclosures + drives in 10 minutes that i didn't know i had |
22:14
🔗
|
yipdw |
ha |
22:25
🔗
|
tpw_rules |
lol just 6TB lying around |
22:27
🔗
|
* |
closure points to shard2 and shard3 |
22:33
🔗
|
db48x |
we should just install a cron job |
22:39
🔗
|
db48x |
EDITOR="cat cron.example >>$1" crontab -e |
23:22
🔗
|
|
zottelbey has quit IRC (Remote host closed the connection) |
23:55
🔗
|
closure |
tpw_rules: still at the top of http://iabak.archiveteam.org/stats/SHARD1.expireleaderboard |