[00:28] *** thunk has quit (http://www.kiwiirc.com/ - A hand crafted IRC client) [00:49] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [01:09] *** wp494_ (~wickedpla@[redacted]) has joined #internetarchive.bak [01:09] *** wp494_ has quit (Excess Flood) [01:09] *** DFJustin has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** Start has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** csssuf has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** garyrh has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** trs80 has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** wp494 has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** goekesmi has quit (ircd.shaw.ca irc.shaw.ca) [01:09] *** ryang has quit (ircd.shaw.ca irc.shaw.ca) [01:10] *** wp494_ (~wickedpla@[redacted]) has joined #internetarchive.bak [01:17] *** ryang_ (uid10904@[redacted]) has joined #internetarchive.bak [01:20] *** Start_ (~Start@[redacted]) has joined #internetarchive.bak [01:23] *** thunk has quit (http://www.kiwiirc.com/ - A hand crafted IRC client) [01:23] *** goekesmi_ (~goekesmi@[redacted]) has joined #internetarchive.bak [01:23] *** Start_ has quit (Client Quit) [01:24] *** Start_ (~Start@[redacted]) has joined #internetarchive.bak [01:29] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [01:34] *** garyrh (garyrh@[redacted]) has joined #internetarchive.bak [01:35] *** svchfoo1 gives channel operator status to garyrh [01:35] *** Start_ is now known as Start [01:36] *** Start has quit (Quit: Disconnected.) [01:39] *** trs80 (~trs80@[redacted]) has joined #internetarchive.bak [01:42] *** trs80 has quit (ircd.shaw.ca irc.shaw.ca) [01:47] *** londoncal has quit (Remote host closed the connection) [02:00] *** trs80 (~trs80@[redacted]) has joined #internetarchive.bak [02:04] *** trs80 has quit (ircd.shaw.ca irc.shaw.ca) [02:13] *** thunk has quit (http://www.kiwiirc.com/ - A hand crafted IRC client) [02:33] *** zottelbey has quit (Remote host closed the connection) [03:19] *** Start (~Start@[redacted]) has joined #internetarchive.bak [03:20] *** svchfoo1 gives channel operator status to Start [04:05] *** VADemon has quit (Quit: left4dead) [04:45] *** bzc6p__ is now known as bzc6p [05:10] *** DFJustin (DopefishJu@[redacted]) has joined #internetarchive.bak [05:11] *** svchfoo1 gives channel operator status to DFJustin [05:44] This sounds great. [06:26] *** Disconnected (Connection reset by peer). [06:27] *** Now talking on #internetarchive.bak [06:27] *** Topic for #internetarchive.bak is: http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK | #archiveteam [06:27] *** Topic for #internetarchive.bak set by chfoo!~chris@[redacted] at Wed Mar 4 18:38:46 2015 [06:36] *** johtso has quit (Quit: Connection closed for inactivity) [06:39] *** Disconnected (Connection reset by peer). [06:40] *** Now talking on #internetarchive.bak [06:40] *** Topic for #internetarchive.bak is: http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK | #archiveteam [06:40] *** Topic for #internetarchive.bak set by chfoo!~chris@[redacted] at Wed Mar 4 18:38:46 2015 [06:41] *** cloudmons (~quassel@[redacted]) has joined #internetarchive.bak [06:45] *** chfoo has quit (Ping timeout: 512 seconds) [07:26] *** ENDING LOGGING AT Mon Mar 16 02:26:56 2015 [07:27] *** BEGIN LOGGING AT Mon Mar 16 02:27:41 2015 [07:31] *** enkiv2 has quit (Read error: Operation timed out) [07:38] *** Control-S (~Ctrl-S@[redacted]) has joined #internetarchive.bak [07:39] *** ENDING LOGGING AT Mon Mar 16 02:39:36 2015 [07:40] *** BEGIN LOGGING AT Mon Mar 16 02:40:23 2015 [07:45] *** Control-S is now known as Ctrl-S [07:45] *** Ctrl-S has quit (Read error: Operation timed out) [07:58] *** niyaje has quit (Ping timeout: 600 seconds) [09:03] *** enkiv2 (~john@[redacted]) has joined #internetarchive.bak [09:30] *** londoncal (~londoncal@[redacted]) has joined #internetarchive.bak [10:01] *** zottelbey (~zottelbey@[redacted]) has joined #internetarchive.bak [10:13] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [10:14] *** thunk has quit (Client Quit) [10:39] *** bzc6p_ (~bzc6p@[redacted]) has joined #internetarchive.bak [10:42] *** bzc6p has quit (Ping timeout: 369 seconds) [11:12] *** niyaje (~niyaje@[redacted]) has joined #internetarchive.bak [11:40] *** niyaje has quit (Ping timeout: 600 seconds) [12:40] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [12:53] *** thunk has quit (http://www.kiwiirc.com/ - A hand crafted IRC client) [13:21] *** wp494_ is now known as wp494 [13:46] *** Start has quit (Disconnected.) [14:10] *** Start (~Start@[redacted]) has joined #internetarchive.bak [14:10] *** svchfoo1 gives channel operator status to Start [14:13] *** Start has quit (Client Quit) [14:23] *** goekesmi_ has quit (Remote host closed the connection) [14:57] is the updated census available? [15:02] *** Start (~Start@[redacted]) has joined #internetarchive.bak [15:37] *** johtso (uid563@[redacted]) has joined #internetarchive.bak [15:49] *** You are now known as chfoo [15:52] *** Start has quit (Disconnected.) [15:57] *** Start (~Start@[redacted]) has joined #internetarchive.bak [15:58] *** Start has quit (Read error: Connection reset by peer) [15:58] *** Start_ (~Start@[redacted]) has joined #internetarchive.bak [16:46] *** Start_ has quit (Disconnected.) [16:51] *** Start (~Start@[redacted]) has joined #internetarchive.bak [16:53] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [16:54] *** thunk has quit (Client Quit) [16:55] *** Start has quit (Read error: Connection reset by peer) [16:55] *** Start (~Start@[redacted]) has joined #internetarchive.bak [16:55] *** Start has quit (Client Quit) [16:58] *** Start (~Start@[redacted]) has joined #internetarchive.bak [16:59] *** Start has quit (Read error: Connection reset by peer) [16:59] *** Start_ (~Start@[redacted]) has joined #internetarchive.bak [17:30] *** londoncal has quit (Remote host closed the connection) [17:34] *** ryang_ has quit (Ping timeout: 617 seconds) [17:34] *** ryang_ (uid10904@[redacted]) has joined #internetarchive.bak [17:40] *** Start_ has quit (Disconnected.) [17:41] I think it is. [17:41] It's on the wiki. [17:43] I only see the one from March 4? [17:48] *** hatseflat has quit (Read error: Connection reset by peer) [17:49] That's the one. Internal file was updated [17:50] oh! thanks [18:08] *** hatseflat (~hatseflat@[redacted]) has joined #internetarchive.bak [18:44] *** Start (~Start@[redacted]) has joined #internetarchive.bak [19:12] *** Start has quit (Disconnected.) [19:21] *** Start (~Start@[redacted]) has joined #internetarchive.bak [19:26] *** Start has quit (Client Quit) [19:31] *** Start (~Start@[redacted]) has joined #internetarchive.bak [20:02] *** Start has quit (Disconnected.) [20:02] *** Start (~Start@[redacted]) has joined #internetarchive.bak [20:23] *** Start has quit (Disconnected.) [20:26] *** Start (~Start@[redacted]) has joined #internetarchive.bak [20:42] ok, making 100k file git repo with usenethistorical+internetarchivebooks collections [20:43] closure: is there an easy way to split these into chunks? or to pick which files go to which chunk? [20:46] which, the collections? [20:46] yeah [20:47] I can import any list of md5 and url. Making ones from collections is easiest [20:48] ok. i'm wondering if it would be easier to have a "standard" chunk size like 500GB, or let each person/computer choose how much data they want [20:49] these repos are not 1 per person [20:49] they contain individual files from the IA, you can get any set you want [20:50] ok. can a client see how many other people have a copy of a particular file? [20:51] yep [20:52] ok cool; [21:04] hmm, fos is slower than my laptop at making these repos [21:04] probably because I have a ssd [21:06] *** Start has quit (Disconnected.) [21:07] *** closure just updated http://git-annex.branchable.com/design/iabackup/ , new simpler design [21:19] annexed files in working tree: 103343 [21:19] size of annexed files in working tree: 2.91 terabytes [21:23] oh is that all [21:29] reading through, this sounds really nice [21:29] "They will also tend to retain the old version", this sounds like a feature [21:30] I think it can work, scaling wise. Bigger questions are, will there be enough people, not too many assholes, and can the IA upload a whole copy of itself in any reasonable time period? [21:30] tending to retain the old version is a feature, unless the old version is grossly illegal [21:31] *** londoncal (~londoncal@[redacted]) has joined #internetarchive.bak [21:34] allowing storage to be offline most of the time is a big plus imo, I'm starting to accumulate drives now which are still reasonably sized (500gb-1tb) but I'm retiring due to replacing with larger sizes or due to having used them for several years [21:34] so I'd be glad to throw a bunch of ia content on them but finding hundreds of gb of free live space is more problematic [21:35] and I would bet other people are in the same boat [21:39] having clients send files to each other seems like the natural answer to the "ia bandwidth go boom" problem but I guess that involves substantial new code [21:40] although, IA is already hosting torrents for most items anyway so maybe it would be practical to automate connecting to those [21:44] *** thunk (4746deec@[redacted]) has joined #internetarchive.bak [21:44] MD5 isn't that weak when the original files and MD5 hashes are fixed. [21:45] the easy ones require the attacker to have at least some control over all the files involved. [22:27] *** thunk has quit (http://www.kiwiirc.com/ - A hand crafted IRC client) [23:03] *** bzc6p__ (~bzc6p@[redacted]) has joined #internetarchive.bak [23:04] *** bzc6p_ has quit (Ping timeout: 369 seconds)