| Time |
Nickname |
Message |
|
00:07
🔗
|
|
niyaje3 has joined #internetarchive.bak |
|
00:09
🔗
|
|
niyaje4 has joined #internetarchive.bak |
|
00:16
🔗
|
|
niyaje3 has quit IRC (Read error: Operation timed out) |
|
00:18
🔗
|
|
svchfoo2 has joined #internetarchive.bak |
|
00:19
🔗
|
|
svchfoo1 sets mode: +o svchfoo2 |
|
00:34
🔗
|
|
svchfoo2 has quit IRC (Quit: Closing) |
|
00:34
🔗
|
|
svchfoo2 has joined #internetarchive.bak |
|
00:35
🔗
|
|
svchfoo3 sets mode: +o svchfoo2 |
|
00:47
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
|
01:01
🔗
|
|
niyaje4 has joined #internetarchive.bak |
|
01:05
🔗
|
|
Start has joined #internetarchive.bak |
|
01:05
🔗
|
|
Start has quit IRC (Client Quit) |
|
01:06
🔗
|
|
Start has joined #internetarchive.bak |
|
01:16
🔗
|
|
wp494 has quit IRC (Ping timeout: 740 seconds) |
|
01:16
🔗
|
|
wp494_ has joined #internetarchive.bak |
|
01:16
🔗
|
|
wp494_ is now known as wp494 |
|
01:29
🔗
|
closure |
ppiixx: cute, I'll bet that's the xml index files, that don't have a known size, so it continues downloading them past the diskreserve setting |
|
01:30
🔗
|
* |
closure fixes.. |
|
01:30
🔗
|
|
tpw_rules has left Evil will always triumph, because good is dumb. |
|
01:30
🔗
|
|
tpw_rules has joined #internetarchive.bak |
|
02:07
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
|
02:36
🔗
|
|
niyaje4 has joined #internetarchive.bak |
|
02:50
🔗
|
|
zottelbey has quit IRC (Remote host closed the connection) |
|
03:32
🔗
|
|
niyaje4 has quit IRC (Ping timeout: 600 seconds) |
|
04:09
🔗
|
|
DopefishJ is now known as DFJustin |
|
04:10
🔗
|
|
svchfoo1 sets mode: +o DFJustin |
|
04:54
🔗
|
|
SketchCow has quit IRC (Read error: Connection reset by peer) |
|
04:54
🔗
|
|
chfoo has quit IRC (Ping timeout: 306 seconds) |
|
04:55
🔗
|
|
Quile_ has joined #internetarchive.bak |
|
04:56
🔗
|
|
espes___ has quit IRC (ny.us.hub irc.teksavvy.ca) |
|
04:56
🔗
|
|
Quile has quit IRC (ny.us.hub irc.teksavvy.ca) |
|
04:56
🔗
|
|
closure has quit IRC (ny.us.hub irc.teksavvy.ca) |
|
04:56
🔗
|
|
espes___ has joined #internetarchive.bak |
|
04:56
🔗
|
|
closure has joined #internetarchive.bak |
|
04:56
🔗
|
|
irc.teksavvy.ca sets mode: +o closure |
|
04:56
🔗
|
|
chfoo has joined #internetarchive.bak |
|
04:58
🔗
|
|
espes___ has quit IRC (ircd.choopa.net irc.teksavvy.ca) |
|
04:58
🔗
|
|
closure has quit IRC (ircd.choopa.net irc.teksavvy.ca) |
|
05:03
🔗
|
|
espes__ has joined #internetarchive.bak |
|
05:05
🔗
|
|
svchfoo3 has quit IRC (Quit: Closing) |
|
05:06
🔗
|
|
svchfoo3 has joined #internetarchive.bak |
|
05:07
🔗
|
|
svchfoo1 sets mode: +o svchfoo3 |
|
05:20
🔗
|
|
closure has joined #internetarchive.bak |
|
05:20
🔗
|
|
svchfoo1 sets mode: +o closure |
|
06:06
🔗
|
|
raylee has joined #internetarchive.bak |
|
06:56
🔗
|
|
yipdw_ has joined #internetarchive.bak |
|
06:58
🔗
|
|
yipdw has quit IRC (Read error: Operation timed out) |
|
07:15
🔗
|
ppiixx |
closure: yeah exactly that |
|
07:27
🔗
|
|
fenn has quit IRC (Read error: Operation timed out) |
|
07:55
🔗
|
|
yipdw_ is now known as yipdw |
|
07:56
🔗
|
|
svchfoo3 sets mode: +o yipdw |
|
08:07
🔗
|
|
Infreq has joined #internetarchive.bak |
|
09:46
🔗
|
|
zottelbey has joined #internetarchive.bak |
|
14:43
🔗
|
|
VADemon has joined #internetarchive.bak |
|
15:03
🔗
|
|
atomotic has joined #internetarchive.bak |
|
16:16
🔗
|
|
atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) |
|
16:55
🔗
|
|
Start has quit IRC (Read error: Connection reset by peer) |
|
16:55
🔗
|
|
Start_ has joined #internetarchive.bak |
|
19:49
🔗
|
|
SN4T14_ has joined #internetarchive.bak |
|
19:55
🔗
|
|
SN4T14 has quit IRC (Ping timeout: 369 seconds) |
|
20:17
🔗
|
|
Start_ is now known as Start |
|
20:54
🔗
|
|
wp494 has quit IRC (LOUD UNNECESSARY QUIT MESSAGES) |
|
20:58
🔗
|
|
wp494 has joined #internetarchive.bak |
|
21:04
🔗
|
|
antomati_ is now known as antomatic |
|
21:31
🔗
|
closure |
tpw_rules: hey, you're at the top of http://iabak.archiveteam.org/stats/SHARD1.expireleaderboard , can you please run iabak periodically? |
|
21:31
🔗
|
tpw_rules |
what does that mean |
|
21:31
🔗
|
tpw_rules |
i haven't installed iabak yet |
|
21:31
🔗
|
closure |
well, you need to |
|
21:32
🔗
|
tpw_rules |
also i think that may be me from when i erased my first archive |
|
21:32
🔗
|
tpw_rules |
lemme check my id |
|
21:32
🔗
|
closure |
I suspect one of them is, and one of them isn't |
|
21:32
🔗
|
closure |
periodically running iabak is how we'll know which repos still exist |
|
21:32
🔗
|
tpw_rules |
ok. how do i check my id? and what does iabak do? |
|
21:33
🔗
|
tpw_rules |
https://archive.org/download/Ttscribe/Ttscribe_files.xml is still darked and preventing me from doing a complete get |
|
21:34
🔗
|
closure |
I suppose that more files will dark from time to time, I'm not worrying about that |
|
21:34
🔗
|
tpw_rules |
i'm running fsck |
|
21:35
🔗
|
closure |
are you running sync after? Are you using the right version of git-annex? iabak takes care of that stuff |
|
21:35
🔗
|
tpw_rules |
yeah and yeah. i meant this second |
|
21:37
🔗
|
tpw_rules |
how often do i come up for expiry? i'll set a cron job |
|
21:39
🔗
|
closure |
also root@katie, root@iashard-de-01, root@iashard-lax-01 |
|
21:39
🔗
|
closure |
currently we're looking at 1 week, may change |
|
21:39
🔗
|
tpw_rules |
is fsck --fast enough? |
|
21:39
🔗
|
closure |
it's too much. iabak has a faster method |
|
21:40
🔗
|
tpw_rules |
yeah i saw it |
|
21:40
🔗
|
tpw_rules |
can i just run iabak in a screen and leave it alone? |
|
21:41
🔗
|
closure |
if you touch IA.BAK/NOMORE, iabak won't check out any more shards at all, and will just do maintenance |
|
21:42
🔗
|
closure |
it doesn't currently keep running forever though.. can run from cron job |
|
21:42
🔗
|
tpw_rules |
ok |
|
21:42
🔗
|
tpw_rules |
is it re-entrant from a cron-job |
|
21:42
🔗
|
closure |
ym run from cron and from command line? |
|
21:42
🔗
|
tpw_rules |
i mean like if i run it daily and it starts a download which takes > day |
|
21:43
🔗
|
tpw_rules |
will it realize it's alrady running and exit |
|
21:43
🔗
|
closure |
not currently |
|
21:43
🔗
|
closure |
people like to run multiple ones to use more BW |
|
21:44
🔗
|
closure |
we could have a iabak-cronjob that is safe that way, and avoids the more expensive stuff |
|
21:44
🔗
|
closure |
ie, doesn't download more shards |
|
21:44
🔗
|
tpw_rules |
oh yeah, ok. forgot it dodn't matter |
|
21:45
🔗
|
tpw_rules |
could you add a 'destination' system to git annex? |
|
21:45
🔗
|
tpw_rules |
so i can say "put x GB here and y GB there" and it will fill them up as much as possible |
|
21:46
🔗
|
tpw_rules |
ideally stuffing with little files to get as close to the max as possible |
|
21:46
🔗
|
tpw_rules |
cause i have a billion hard drives just lying around and i want to attach them to one system |
|
21:46
🔗
|
closure |
that's accomplished by having different clones of the repo in different places |
|
21:46
🔗
|
yipdw |
use lvm :P |
|
21:46
🔗
|
tpw_rules |
i don't want them to duplicate files though |
|
21:47
🔗
|
tpw_rules |
yipdw: the problem with that is that if one drive fails, everything goes down |
|
21:47
🔗
|
yipdw |
zfs in raidz2/z3 mode |
|
21:47
🔗
|
closure |
you can teach git-annex that a repo doesn't want files that are in another repo |
|
21:48
🔗
|
closure |
or you can move files manually from one to another |
|
21:48
🔗
|
tpw_rules |
but they're different sizes and that wastes space on redundancy |
|
21:48
🔗
|
yipdw |
well |
|
21:48
🔗
|
tpw_rules |
i just want this for the couple dozen laptop and desktop drives i have lying around to be useful (and also stress-test if i need one for something) |
|
21:48
🔗
|
closure |
cd shard1; git remote add otherdrvie /other/drive/shard1 ; git annex move --to otherdrive |
|
21:49
🔗
|
yipdw |
it sounds like there's higher-layer solutions available so that's fine, but I'm usually a fan of pushing the physical->logical mapping deeper into the stack so I don't have to care |
|
21:49
🔗
|
tpw_rules |
yipdw: the problem is the physical map is fairly fluid |
|
21:50
🔗
|
|
Quile_ has quit IRC (Read error: Operation timed out) |
|
21:50
🔗
|
|
Quile has joined #internetarchive.bak |
|
21:50
🔗
|
tpw_rules |
and a drive dying killing the entire thing would be a waste of bandwidth and time |
|
21:51
🔗
|
tpw_rules |
closure: how often does the leaderboard update? sync is done |
|
21:53
🔗
|
closure |
only once an hour |
|
22:07
🔗
|
tpw_rules |
so it turns out i actually have spare hard drives out my butt. found four enclosures + drives in 10 minutes that i didn't know i had |
|
22:14
🔗
|
yipdw |
ha |
|
22:25
🔗
|
tpw_rules |
lol just 6TB lying around |
|
22:27
🔗
|
* |
closure points to shard2 and shard3 |
|
22:33
🔗
|
db48x |
we should just install a cron job |
|
22:39
🔗
|
db48x |
EDITOR="cat cron.example >>$1" crontab -e |
|
23:22
🔗
|
|
zottelbey has quit IRC (Remote host closed the connection) |
|
23:55
🔗
|
closure |
tpw_rules: still at the top of http://iabak.archiveteam.org/stats/SHARD1.expireleaderboard |