[01:25] *** beardicus has quit IRC (Read error: Operation timed out) [01:25] *** phuzion has quit IRC (Read error: Operation timed out) [01:25] *** arkiver has quit IRC (Read error: Operation timed out) [01:27] *** GLaDOS has quit IRC (Read error: Operation timed out) [01:30] *** GLaDOS has joined #internetarchive.bak [01:30] *** arkiver has joined #internetarchive.bak [01:30] *** svchfoo3 sets mode: +o arkiver [02:18] *** beardicus has joined #internetarchive.bak [02:18] *** svchfoo3 sets mode: +o beardicus [02:18] *** phuzion has joined #internetarchive.bak [02:19] *** svchfoo3 sets mode: +o phuzion [06:30] *** JesseW has joined #internetarchive.bak [06:30] FYI, the iabak script doesn't work with zsh. This is probably not a problem. [06:36] no, not a huge problem [07:14] db48x: BTW, your commit back a year ago https://github.com/ArchiveTeam/IA.BAK/commit/68cacc3d7eab687e234f43d3876e6898ac31005c broke outofspace [07:27] I do recall testing that [07:30] Hm, strange. [07:30] because I don't think [07:30] if [ which numfmt >/dev/null 2>&1]; then [07:30] makes much sense. [07:32] http://linux.die.net/man/1/test -- shows that the conditional statement doesn't take arbitrary commands like that [07:33] though I see what you mean [07:36] I think it's probably better to just dump the use of numfmt and go with the bytesfromsize form [07:44] I must have been tired when I wrote this [07:44] the B case is slightly wrong [07:44] I declare size to be an array, then don't use it as an array [07:45] *** JesseW has quit IRC (Ping timeout: 370 seconds) [07:45] one caller of bytesFromSize checks for numfmt first, another doesn't [09:31] *** antomati_ has joined #internetarchive.bak [09:31] *** HCross2 has quit IRC (Ping timeout: 246 seconds) [09:34] *** antomatic has quit IRC (Ping timeout: 260 seconds) [09:48] *** atomotic has joined #internetarchive.bak [10:47] rebooted and seem to be getting 30Mbps down, which is better [11:03] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [11:14] *** VADemon has joined #internetarchive.bak [12:20] So, I'd love for us to have a windows port of this. [12:21] I think that could cause a mass of interest/additional people. [12:21] Also may be time to start making a massive list of what items in the collections should generally be saved [12:47] *** atomotic has joined #internetarchive.bak [13:38] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [13:38] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [13:40] *** r3c0d3x has joined #internetarchive.bak [13:51] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [13:52] *** r3c0d3x has joined #internetarchive.bak [14:00] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:01] *** r3c0d3x has joined #internetarchive.bak [14:08] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:10] *** r3c0d3x has joined #internetarchive.bak [14:18] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:19] *** r3c0d3x has joined #internetarchive.bak [14:24] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:25] *** r3c0d3x has joined #internetarchive.bak [14:32] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:33] *** r3c0d3x has joined #internetarchive.bak [14:38] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:39] *** r3c0d3x has joined #internetarchive.bak [14:46] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:47] *** r3c0d3x has joined #internetarchive.bak [14:52] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [14:53] *** r3c0d3x has joined #internetarchive.bak [15:01] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:02] *** r3c0d3x has joined #internetarchive.bak [15:07] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:07] *** r3c0d3x has joined #internetarchive.bak [15:14] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:15] *** r3c0d3x has joined #internetarchive.bak [15:20] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:22] *** r3c0d3x has joined #internetarchive.bak [15:27] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:28] *** r3c0d3x has joined #internetarchive.bak [15:33] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:34] *** r3c0d3x has joined #internetarchive.bak [15:41] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:41] *** r3c0d3x has joined #internetarchive.bak [15:45] *** r3c0d3x_ has joined #internetarchive.bak [15:46] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:50] *** r3c0d3x_ has quit IRC (Ping timeout: 260 seconds) [15:51] *** r3c0d3x has joined #internetarchive.bak [15:56] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [15:57] *** r3c0d3x has joined #internetarchive.bak [16:02] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:02] *** r3c0d3x has joined #internetarchive.bak [16:09] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:10] *** r3c0d3x has joined #internetarchive.bak [16:15] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:15] *** r3c0d3x has joined #internetarchive.bak [16:16] *** JesseW has joined #internetarchive.bak [16:19] *** r3c0d3x_ has joined #internetarchive.bak [16:20] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:24] *** r3c0d3x_ has quit IRC (Ping timeout: 260 seconds) [16:27] *** svchfoo3 has quit IRC (Read error: Connection reset by peer) [16:28] *** r3c0d3x has joined #internetarchive.bak [16:30] *** svchfoo3 has joined #internetarchive.bak [16:30] *** svchfoo1 sets mode: +o svchfoo3 [16:33] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:38] *** r3c0d3x has joined #internetarchive.bak [16:44] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:45] *** r3c0d3x has joined #internetarchive.bak [16:49] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [16:52] *** JesseW has quit IRC (Ping timeout: 370 seconds) [16:54] *** r3c0d3x has joined #internetarchive.bak [16:59] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:04] *** r3c0d3x has joined #internetarchive.bak [17:09] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:13] *** r3c0d3x has joined #internetarchive.bak [17:19] *** r3c0d3x_ has joined #internetarchive.bak [17:19] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:24] *** r3c0d3x_ has quit IRC (Ping timeout: 260 seconds) [17:29] *** r3c0d3x has joined #internetarchive.bak [17:34] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:37] *** r3c0d3x has joined #internetarchive.bak [17:42] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:43] *** r3c0d3x has joined #internetarchive.bak [17:48] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:51] *** r3c0d3x has joined #internetarchive.bak [17:55] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [17:59] *** r3c0d3x has joined #internetarchive.bak [18:03] *** r3c0d3x_ has joined #internetarchive.bak [18:04] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [18:08] *** r3c0d3x_ has quit IRC (Ping timeout: 260 seconds) [18:08] *** r3c0d3x has joined #internetarchive.bak [18:13] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [18:13] *** r3c0d3x has joined #internetarchive.bak [18:18] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [18:23] *** r3c0d3x has joined #internetarchive.bak [18:23] *** JesseW has joined #internetarchive.bak [18:28] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [18:34] SketchCow: The census work I've done may be helpful in figuring out what is good to save. [18:34] Or, keep duplicates of, I mean. [18:35] I can't help with the Windows port, though. :-( [18:37] I'm going to write some tests for the shell scripts (mainly because I think it would be fun). I'll make a PR. If this is actively unwanted, let me know. [19:05] *** r3c0d3x has joined #internetarchive.bak [19:08] The initial confirmation that you want to continue interacts badly with the cleanup trap [19:09] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [19:11] *** r3c0d3x has joined #internetarchive.bak [19:15] *** r3c0d3x has quit IRC (Ping timeout: 260 seconds) [19:16] https://github.com/ArchiveTeam/IA.BAK/pull/45 [19:19] quick question - general consensus on getting things from ISP caches instead of the IA? [19:33] clarify? what ISP caches? [19:34] AFAIK, the hashes of the content are checked, so where you get copies of the full content shouldn't matter much [19:35] My ISP runs a large set of caches on their network, managed to block them though by firewalling off IPs [19:40] Well, the initial repo you clone ( https://github.com/ArchiveTeam/IA.BAK ) contains a list of repo URLs (one for each shard) which look like this: SHARD1@iabak.archiveteam.org:shard1 [19:41] Those repos contain git-annex references to the actual content. [19:41] Which can be gotten either directly from IA, or from anywhere else. [19:42] ah [20:09] HCross: useful link: https://git-annex.branchable.com/design/iabackup/ [21:02] *** patrickod has joined #internetarchive.bak [21:44] *** closure has joined #internetarchive.bak [21:49] *** yipdw has quit IRC (Quit: No Ping reply in 180 seconds.) [21:49] *** svchfoo1 sets mode: +o closure [21:50] *** yipdw has joined #internetarchive.bak [22:15] *** yipdw has quit IRC (Read error: Operation timed out) [22:22] *** yipdw has joined #internetarchive.bak [23:11] bah [23:12] missed my package while dozing on the couch [23:12] :/ [23:12] good nap though [23:13] How do I see myself on http://iabak.archiveteam.org ? [23:14] click on "Top 25 contributors" [23:15] What name is it showing there? [23:15] ahh, there I am [23:15] 43408754505: 5c56bf8a-a4a6-44c4-a9d7-3a17b9dd7e67 -- root@harry-Inspiron-400:/IA.BAK/shard4 [23:16] 340GB to go [23:17] all ive got is shard9, as shard4 messed up mid download (aka I pulled the plug on the wrong PC late at night) [23:17] see also http://iabak.archiveteam.org/client/d1e0ad71e37566a62a52266aa800c8df24947fd1.html [23:18] http://iabak.archiveteam.org/client/d1e0ad71e37566a62a52266aa800c8df24947fd1.html hmm [23:18] Can someone remove shard4 from me? [23:18] why? [23:19] I dont actually have it downloaded [23:19] You should be able to remove it yourself, I think... [23:19] had to delete it and start again [23:19] it'll go away on it's own [23:19] its not on my client, but still showing on the site [23:19] ah k [23:19] thanks [23:19] HCross: here's an example of what it will look like, I think: http://iabak.archiveteam.org/client/75c240639a70ec6bd740e48ecf304b66b0d233e3.html [23:20] Is the server code available somewhere? I'm curious to see how it's generating the reports... [23:21] ok [23:21] JesseW: https://github.com/ArchiveTeam/IA.BAK/tree/server [23:21] just need to work out how to make it say "HCross" and not "me" [23:22] heh -- hidden in a separate branch. Interesting. [23:26] HCross: use the change-email script to change the email address you have registered with the server [23:26] what do I change it too? [23:27] ahh [23:27] because my email is me@blah thats where its getting it [23:28] yep [23:30] hmm, might have to setup an address for it then [23:33] JesseW: so this pull request [23:34] how do I reproduce this problem you're fixing? [23:34] db48x: Run ./iabak in a directory with no shards, and press Ctrl-C at the initial prompt. [23:35] It will print "Cleaning up..." but won't finish. And if you press Ctrl-C again, it will just print "Cleaning up..." again. The only way to exit is to press Enter, then Ctrl-C partway through it downloading git annex (or whatever it ends up doing next). [23:36] ah, I see [23:37] ~35 hours for my download [23:37] Something about the interaction of `read` and `trap` [23:38] yea, looks good [23:38] * JesseW is stuck in my efforts at test writing by needing to isolate the test env from system binaries (yes, I know this is possible, I just need to wrap my head around it) [23:47] * JesseW is now less stuck