[00:31] *** mariusz has joined #internetarchive.bak [00:32] fatal: Out of memory? mmap failed: Cannot allocate memory [00:32] git-annex-shell: git-shell failed [00:33] any ideas? [00:39] *** Apathy_ has quit IRC (Quit: OOOOoooooooooo................) [00:41] *** Apathy has joined #internetarchive.bak [01:01] mariusz: which repo? [01:01] shard [01:17] *** wp494 has quit IRC (Ping timeout: 306 seconds) [01:19] 8 [01:20] but this also happend when i ran cron job [01:26] *** primus104 has quit IRC (Leaving.) [01:32] there was a server problem that broke 3. perhaps 8 is now suffering the same affliction [01:39] That's the same symptoms 3 had [01:41] I think we're finding exciting new bugs [02:13] If so we might start seeing it on other shards too :-( [02:14] oh well, nobody expected this to be bug free [02:15] Someone should tell db-whatever-his-name-is next time they see him [02:20] dee bee fourty eight ex? [02:21] *** hatseflat has quit IRC (Read error: Operation timed out) [02:23] *** hatseflat has joined #internetarchive.bak [05:22] *** wickedpla has joined #internetarchive.bak [05:22] *** wickedpla is now known as wp494 [05:23] *** wp494 is now known as wp494_ [05:23] *** wp494_ is now known as wp494 [08:08] *** primus104 has joined #internetarchive.bak [08:18] *** db48x has joined #internetarchive.bak [08:31] *** primus104 has quit IRC (Leaving.) [08:57] 03registrar 05master 242b49f 06other 10SHARD9/pubkeys registration of sunny on SHARD9 [09:04] *** primus104 has joined #internetarchive.bak [10:12] *** primus104 has quit IRC (Leaving.) [11:39] *** VADemon has joined #internetarchive.bak [12:19] *** zz_CyberJ is now known as CyberJaco [12:52] 03registrar 05master 29b3bae 06other 10SHARD7/pubkeys registration of sunny on SHARD7 [13:06] 03registrar 05master 8018d79 06other 10SHARD8/pubkeys registration of sunny on SHARD8 [13:08] *** mariusz_ has joined #internetarchive.bak [13:32] 03registrar 05master a3b8962 06other 10SHARD9/pubkeys registration of sunny on SHARD9 [13:33] *** Apathy has quit IRC (Excess Flood) [13:33] *** Apathy has joined #internetarchive.bak [13:39] shard9?? what's the point if we can't have current 8 complete? [13:41] Bandwidth is an issue. Might as well get on with downloading *something*? [13:42] I'm downloading on 2,5,and 6 and would be on 3 and 8 if they were available [13:44] ok.. any idea if/when 3 and 8 will be back? [13:45] When the bug gets fixed? Might have to wait for closure to get back from holiday? [14:19] was it figured out? [14:19] i'm pretty much out of space tho :( [14:30] *** primus104 has joined #internetarchive.bak [14:32] hm. i'm starting to get out of mem on shard6 [14:32] hmmm [14:42] yeap, it's fked [14:45] *** toad1 has joined #internetarchive.bak [14:46] I'm surprised it's happening on all the shards about at onceish. Maybe it's a recent bug in git-annex? [14:47] *** toad2 has quit IRC (Read error: Operation timed out) [14:55] the only question is how hard it will be to recover from that [15:08] *** mariusz_ has quit IRC (Read error: Operation timed out) [17:44] So, just to answer the question. [17:45] We have SHARD9 set up "just in case" for people who register enough and have enough space that they flush out their copy of the needed space. [17:45] I absolutely think we have a round of bugfixes awaiting db48x and closure [19:25] 03registrar 05master f6f1110 06other 10SHARD7/pubkeys registration of chris on SHARD7 [19:30] *** RX14 has joined #internetarchive.bak [19:31] i'm having an issue running the internetarchive.bak script: https://i.imgur.com/9R0B5lq.png [19:33] RX14: that's normal. some files are no longer available publicly to be backed up. you can ignore them [19:33] it looks to be happening for every file, unless it doesn't log every file [19:34] and if it is normal, you shouls probably put a note on the wiki [19:34] you probably downloaded a bunch of working ones and now it's retrying all the broken ones [19:34] give it time [19:35] i just started it actually [19:35] ah, you must not have shuf [19:35] but it will do it eventually [19:35] aha [19:35] that partucular one has a bunch at the beginning [19:35] this looks better [19:36] progress bars and wget? [19:36] after many attempts [19:36] yup [19:36] now i need more concurrent downloads [19:36] there [19:37] it won't print anything if it downloads concurrently. i like to run a bunch at once so i get a screen of progress bars [19:37] looks more important that way [19:37] that's annoying, is that a git-annex thing [19:37] yeah [19:37] what os do you use? [19:37] i can view nethogs [19:37] linux [19:37] my dedi [19:37] i like to use iftop [19:37] never heard of that [19:37] it's like top but for internet [19:38] why is apt so sloooow [19:38] yeah, iftoop looks neat [19:39] if you pass -B -m 50M [19:39] then it will show in bytes/sec and keep the scale fixed at 50Mbits/sec [19:40] why would you want to fix the scale? [19:40] i liked it to see how much of my internet i'm using total [19:40] lets fix it at 1gbps then [19:40] how much disk do you have [19:40] 900GB [19:40] ahh [19:41] -f 'net 207.241' [19:41] ? [19:41] will make it only count packets coming/going to archive'org domains [19:41] ok [19:41] (not quite but enough for the purposes) [19:41] and, ANNEXGETOPTS doesn't seem to work [19:41] where do you put ANNEXGETOPTS [19:41] yeah it doe [19:41] s [19:42] in a file in the IA.BAK directory [19:42] like echo -J100 > ANNEXGETOPTS [19:42] i did that [19:42] i only see 1 connection in iftop [19:42] did you restart the client? [19:42] and i see progress bars [19:42] also install shuf [19:42] tpw_rules, i did [19:42] what script are you running? [19:42] ./iabak [19:42] yup [19:42] i ctrl-c [19:42] hm [19:43] doesn't seem to have stopped every process though [19:43] do you have git annex installed on your system though [19:43] yeah it doesn't do that [19:43] i don't [19:43] ok [19:43] can you paste the whole log from you doing ./iabak to it downloading? [19:45] http://hastebin.com/ijicexized [19:46] how many concurrent downloads did you say? [19:46] 7 [19:46] it's just -J7 in the file [19:46] so cat ANNEXGETOPTS is "-J7" [19:46] hm [19:46] yup [19:46] i think... [19:46] it might have been 5 [19:46] no newline? [19:47] well echo prints a newline [19:47] so i guess it does [19:47] hm. it should work tho [19:48] echo -n '-J5' > ANNEXGETOPTS [19:48] then cat ANNEXGETOPTS should be -J5 [19:48] also install shuf [19:48] i have suf [19:49] it's not detecting it [19:49] i just typed shuf [19:49] it's a command [19:49] hm [19:49] do you use a different shell? [19:49] zsh [19:49] oh, it may hate you [19:49] i tried without the newline [19:49] it worked [19:50] i think [19:50] it has the progress bar [19:50] shuf may be a builtin [19:50] in zsh [19:50] but it has 5 connections [19:50] oh and it's back down to 1 [19:50] that could be because you don't have shuf. also it may just be downloading five files from one domain [19:50] 500KB/s [19:50] does /usr/bin/shuf exist? [19:51] oh [19:51] one archive.org server is being really slow [19:51] it's probably finishing the file before you ctrl+c it [19:51] what does 'which shuf' give you? [19:51] /usr/bin/shuf [19:52] is /bin/sh a link to zsh somehow? [19:52] let me try running it in bash [19:52] sh seems to be a link to dash [19:52] wth [19:53] it may not work quite right wiht that [19:54] i ran it with bash iabak [19:54] and it's still not in parallel [19:54] maybe [19:54] has it downloaded more than one file? [19:54] try commenting out line 100 in iabak-helper [19:54] htop says one wget binary [19:55] is it at least detecting shuf? [19:56] it's downloading [19:56] but only at 140Mbps [19:56] awwww [19:56] only lol [19:56] my home connection peaks at about 120 [19:56] but i have no more hard drives to shove in to my pc [19:56] i should run it from home too... [19:56] about 40TB as of now [19:57] i'm runnign this on my dedi [19:58] ah, i lied a little [19:58] 25TB used [19:58] 28TB free [19:58] total* [19:58] i don't have that much storage [19:58] i do <3 [19:59] but now i am out [20:00] wget seems to be in parallel now [20:00] i'm using a program called mhddfs to mount a bunch of filesystems as one, kind of like raid but not [20:00] after i commented out your line [20:00] so i'm up to /dev/sdn [20:01] nice [20:02] i should put this on my nas [20:02] but actually I shouldn't [20:02] i had a shit-ton of hard drives laying around so i just bought really cheap enclosures on ebay and i have like 15 usb drives hanging off my poor nas [20:02] i considered the same [20:02] i have a ghetto nas [20:03] pentium 4 [20:03] got some shitty PCI (not e) raid cards [20:03] no raid [20:03] used them to add sata support [20:03] ah, mine's real [20:03] ran btrfs on the random collection of disks [20:03] only 3 disks in there right noew [20:04] zfs master race here (at least for the real nas part) [20:04] i should have gone for zfs [20:04] the neat thing of mhddfs is that if one of the disks implodes, i'll only lose the junk on that disk [20:04] btrfs crashes all the time [20:04] because each disk has an ext4 filesystem and it just combines the files together [20:04] tpw_rules, btrfs is the same [20:05] is it? [20:05] are there archive servers outside the us? [20:05] no [20:05] i think they're all in one datacenter [20:05] because my dedi is in france [20:05] which is why we are trying to back them up [20:05] online.net have really good dedi plans [20:05] a given file is only one one server anyway [20:06] nah they got the it down pretty well [20:08] i wish i had a load of disks lying around [20:08] well i do have a load of disks lying around [20:09] except they are all unles 150GB [20:09] so not that useful [20:13] tpw_rules, so can you just run iabak many times? [20:14] yeah [20:14] and they don't conflict [20:14] yeah [20:19] http://i.imgur.com/EqpSmQr.png [20:19] this is fun [20:19] that's what i did [20:20] it's strange how one is 16MB/s and the rest are flagging [20:40] dash> /bin/sh is usually dash nowadays on linux; or posh, or another small shell [20:41] well i'm up and running so... it wroks now? [23:23] *** RX14 has quit IRC (Remote host closed the connection)