[01:07] *** RedType has quit IRC (Ping timeout: 252 seconds) [01:07] *** bwn has quit IRC (Read error: Operation timed out) [01:23] *** RedType has joined #archiveteam-bs [01:35] *** primus104 has quit IRC (Leaving.) [01:43] *** zyphlar__ is now known as zyphlar [02:01] *** DMackey- has joined #archiveteam-bs [02:06] *** DMackey has quit IRC (Read error: Operation timed out) [02:33] *** ens has left [03:39] *** remsen has joined #archiveteam-bs [04:05] *** vitzli has joined #archiveteam-bs [04:14] *** remsen has quit IRC (Read error: Operation timed out) [04:24] *** SilSte has quit IRC (Read error: Operation timed out) [04:25] *** aaaaaaaaa has quit IRC (Leaving) [04:38] *** SilSte has joined #archiveteam-bs [05:10] *** vitzli has quit IRC (Leaving) [05:41] Anyone know what files like tmp-wpull-warcsesrsp-4p603of8.tmp and tmp-wpull-resp_cb-frl2x0wd.tmp are? [05:41] I've got 8 really big ones (8–9gb) [05:42] *** remsen has joined #archiveteam-bs [05:51] *** vitzli has joined #archiveteam-bs [05:51] *** remsen has quit IRC (Leaving) [05:51] *** remsen has joined #archiveteam-bs [05:55] If I get hit by the bus, you better buy a new fucking bus [05:56] https://anachips.files.wordpress.com/2014/12/canadian-bus-crash_o_1203746.jpg [06:01] *** bwn has joined #archiveteam-bs [06:09] *** remsen has quit IRC (Leaving) [06:09] *** remsen has joined #archiveteam-bs [06:11] *** primus104 has joined #archiveteam-bs [06:45] *** nightpool has quit IRC (Ping timeout: 310 seconds) [07:05] *** primus104 has quit IRC (Leaving.) [07:12] *** Sk1d has joined #archiveteam-bs [07:15] Hah [07:33] *** DMackey- has quit IRC (Read error: Connection reset by peer) [07:33] *** DMackey has joined #archiveteam-bs [07:42] Mmmh, I'd like a private train.. just like HCross has [08:04] argh so many Cyber Monday emails [08:04] it's so much better if you read it with the true definition of cyber, though [08:05] *** brayden has joined #archiveteam-bs [08:05] *** swebb sets mode: +o brayden [08:05] cyber ;) ;) [08:05] asl etc [08:10] *** BlueMaxim has quit IRC (Quit: Leaving) [08:10] *** BlueMaxim has joined #archiveteam-bs [08:15] *** WinterFox has joined #archiveteam-bs [08:17] Does the internet archive do anything special to seed so many torrents? My transmission daemon started to struggle at 409 [08:17] Started getting errors about too many files open and after I added one torrent with a lot of web seeds the daemon would hit 100% cpu usage and segfault [08:18] Does it even seed? I had the impression that it just takes files from the torrent and disconnects [08:19] It does stop seeding the torrent you upload with but then makes its own one for every collection [08:19] Which is a fair bit by now [08:19] I *think* one of the problems I had was hitting the limit of file descriptors the user could use [08:25] Oh, that of course impacts how many torrents the client will handle [08:25] if you have many with bazillions of files.. then yeah, you ain't gonna be able to load all that many [08:26] most of my 1200+ torrents only have one or five large files in 'em [08:41] *** arkiver2 has joined #archiveteam-bs [08:43] *** Microguru has joined #archiveteam-bs [09:07] *** schbirid has joined #archiveteam-bs [09:15] ersi, The internet archive has stupidly large amounts of files. I guess its not all on one daemon though [09:31] *** primus104 has joined #archiveteam-bs [10:19] SketchCow: i'm archiving Marty Roberts ''Staying Real in Israel" podcast [10:19] i noticed it in your podcast vault items [10:22] *** bwn has quit IRC (Read error: Operation timed out) [10:23] that guys blog may have all of the mp3 [10:23] the blog is in archivebot right now [10:23] i'm brute forcing the mp3 paths [10:37] *** bwn has joined #archiveteam-bs [10:58] *** vitzli has quit IRC (Quit: Leaving) [11:36] *** bwn has quit IRC (Read error: Connection reset by peer) [11:37] *** bwn has joined #archiveteam-bs [11:55] *** arkiver2 has quit IRC (Quit: Nettalk6 - www.ntalk.de) [12:03] *** WinterFox has quit IRC (Remote host closed the connection) [12:09] *** DMackey- has joined #archiveteam-bs [12:12] *** DMackey has quit IRC (Ping timeout: 310 seconds) [12:48] webseeds :) [13:01] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [13:10] *** vitzli has joined #archiveteam-bs [13:12] *** primus104 has quit IRC (Leaving.) [13:20] *** vtyl has quit IRC (Read error: Operation timed out) [13:22] *** lytv has joined #archiveteam-bs [13:44] *** kyan has quit IRC (Leaving) [14:13] *** remsen has quit IRC (Read error: Connection reset by peer) [14:25] *** signius has quit IRC (Ping timeout: 310 seconds) [14:36] *** signius has joined #archiveteam-bs [14:42] *** primus104 has joined #archiveteam-bs [14:44] *** Microguru has quit IRC (Read error: Connection reset by peer) [15:03] *** brayden has quit IRC (Quit: Leaving) [15:03] *** brayden has joined #archiveteam-bs [15:03] *** swebb sets mode: +o brayden [15:04] *** primus104 has quit IRC (Leaving.) [15:21] *** remsen has joined #archiveteam-bs [15:51] *** Start has quit IRC (Quit: Disconnected.) [15:53] *** primus104 has joined #archiveteam-bs [16:12] *** nightpool has joined #archiveteam-bs [16:33] Hi, working my ass off in the information cube. [16:34] I of course was going to tell him, but if he comes back, ximm@archive.org designed the torrent system (took him a year) and he worked heavily with Bittorrent, Inc, to engineer it. [16:34] SketchCow, did you say weather we could go ahead with that 19TB of mouse genomes? [16:50] *** nightpool has quit IRC (Ping timeout: 183 seconds) [16:56] *** atlogbot has quit IRC (Ping timeout: 369 seconds) [16:56] *** swebb has quit IRC (Read error: Operation timed out) [17:05] HCross: i would not bother with huge academic datasets [17:15] *** Start has joined #archiveteam-bs [17:15] *** swebb has joined #archiveteam-bs [17:16] *** atlogbot has joined #archiveteam-bs [17:17] *** nightpool has joined #archiveteam-bs [17:26] ok [17:26] HCross: I guess this is a better channel to ask that question. What's your upstream? [17:27] 12Mbps but throttles [17:27] That's better than I've got at home. I've got something like 1.5mbps, which sucks. [17:28] I wont get that to the IA though [17:29] ill pop them on my external HDD and try tomorrow on a faster link [17:29] HCross: Just create a torrent, and let IA pull it that way if you're concerned about the connection being interrupted. [17:32] Can we please set this to the FTP project logo? http://www.courthousenews.com/2014/04/03/ftp-police.jpg This man is clearly passionate about the file transfer protocol. [17:34] Ill do the torrent. My British ISP isnt known as the best for reliability [17:40] Gotcha. Are you ware of how the whole torrent system works? [17:42] yep [17:42] they traffic shape horrifically when you upload a lot [17:43] we are talking a 12Mbps link becomes under 3Mbps after 2.25GB in an hour [17:51] lol the UK [17:51] more like "lol Liberty Global" [17:51] lolk [17:51] russian lego catalogs: https://archive.org/details/godaneinbox?sort=-publicdate&and[]=subject%3A%22Katalog%20Lego%22 [17:51] United LOL [18:06] https://archive.org/details/murzilka-1924-01 [18:12] phuzion, thanks. The torrent is now working and doing ok at speed [18:15] *** sigkell has quit IRC (Ping timeout: 252 seconds) [18:15] *** antonizoo has quit IRC (Ping timeout: 252 seconds) [18:16] *** signius has quit IRC (Read error: Operation timed out) [18:16] *** sigkell has joined #archiveteam-bs [18:17] *** antonizoo has joined #archiveteam-bs [18:18] *** Kazzy has quit IRC (Read error: Operation timed out) [18:18] Sure enough, Hi traffic shaping [18:19] *** is-_ has quit IRC (Read error: Operation timed out) [18:19] *** Kazzy has joined #archiveteam-bs [18:21] *** is- has joined #archiveteam-bs [18:22] godane, are you grabbing magazines from rutracker.org? [18:24] i'm grabbing some [18:25] I wanted to offer grabing "Vokrug sveta", but it's already in IA [18:26] yes but looks incomplete [18:27] also some of items look odd [18:27] like this one: https://archive.org/details/vokrugsveta1999-01 [18:27] HCross: Cool, glad to see it's working. [18:29] except it seems to be screwing with my upstream network atm [18:29] packets are taking 100ms to leave my street [18:31] *** signius has joined #archiveteam-bs [18:31] They're leaving your LAN but hitting latency at the second or third hop? [18:33] second hop. Its my pile of rubbish cable co [18:33] Throw some mains current onto the line and incentivize them to upgrade infrastructure :) [18:33] Rubbish Cable Company, just a bit better than Shag in a Skip Satellite Internet [18:33] jk don't do that. [18:34] Put it this way, it was owned by Richard Branson at one point [18:35] *cough* VM [18:36] yep. Stuck the torrent file on 2 EU seedboxes and using them to take the transatlantic load off [18:36] VM [18:37] *** remsen has quit IRC (Read error: Operation timed out) [18:38] hah good internet speed, just unreliable at times [18:39] Yep - and also lovers of price increases [18:39] *** Start has quit IRC (Quit: Disconnected.) [18:40] wont touch them again with a barge pole [18:40] godane, that's silly, the item is in fb2 compressed format, hence .zip extension [18:40] rather have slower internet than intermittent [18:42] The IA seems to be coping well with the 4 things that are feeding the torrent [18:43] its faster for me to seed to a server in the EU and then have the IA suckle off that [18:43] using fb2 for that kind of media (photo-heavy magazines/books) is silly [18:51] *** Start has joined #archiveteam-bs [18:53] just had the shout of "STOP USING ALL THE INTERNET" come from downstairs [18:55] funny, I never have that here.. [18:55] :P [18:55] haha [18:55] Stuck it on pause until people have gone to bed [18:55] my speed battle is vs the on demand services [18:56] My speed battle is versus my father and his world of warships [18:56] *** Barry has quit IRC (Ping timeout: 310 seconds) [18:56] *** Barry has joined #archiveteam-bs [18:58] docstoc has 0 items left [18:59] requeued 50000 items [19:13] *** vitzli has quit IRC (Leaving) [19:17] *** Start has quit IRC (Quit: Disconnected.) [19:20] *** Start has joined #archiveteam-bs [19:24] *** Start has quit IRC (Client Quit) [20:23] so i'm ripping Network World from google books [20:23] i have 1986 to 1988 so far [20:29] looks like guys only have 23 Network World files [20:30] i think i can fill that gap so you guys can have a collection [20:32] matthusb_, I think you are needed in #effteepee [20:32] matthusby: do you have a log for iaurl:archiveteam_ftp_items_2015112801:az3.engr.oregonstate.edu_110 ? [20:33] of the ftp grab [20:33] the size of your item was way under the size it should be [20:33] or you had some really really good compression [20:37] *** bwn has quit IRC (Ping timeout: 255 seconds) [20:45] *** Marcelo has joined #archiveteam-bs [20:49] *** Start has joined #archiveteam-bs [20:58] crappy provider but some cheap vms at http://oneprovider.com/onecloud/ssd-virtual-servers today [21:04] *** Muad-Dib has joined #archiveteam-bs [21:14] *** bwn has joined #archiveteam-bs [21:18] *** BlueMaxim has joined #archiveteam-bs [21:22] *** aaaaaaaaa has joined #archiveteam-bs [21:26] *** schbirid has quit IRC (Quit: Leaving) [21:33] Tracker having issues? Error 500 a lot overhere [21:36] whoa, the munin graphs look bad. [21:36] I am going quite hard, do I need to slow down? [21:39] I'm not a sysadmin, but the activity on it is definitely different the normal [21:40] xmc: may want to check the tracker. [21:43] HCross: vmedia? ouch [21:43] yep [21:44] thought i had it bad at 80/20 with no caps [21:44] *** kyan has joined #archiveteam-bs [21:47] aaaaaaaaa: Huh, yeah there's definitely something up since friday morning. [21:52] I wonder if the software is starting to creak under the load or something like that. [21:58] What I noticed that was that disk latency jumped and throughput flatlined. [21:59] * phuzion out [22:06] *** nightpool has quit IRC (Read error: Operation timed out) [22:09] *** nightpool has joined #archiveteam-bs [22:09] *** DMackey has joined #archiveteam-bs [22:11] *** DMackey- has quit IRC (Ping timeout: 310 seconds) [22:16] *** Start has quit IRC (Quit: Disconnected.) [22:28] *** kyan has quit IRC (Quit: This computer has gone to sleep) [22:31] *** nightpool has quit IRC (Ping timeout: 252 seconds) [22:48] *** nightpool has joined #archiveteam-bs [22:49] What's going on with the tracker? [23:01] *** balrog has quit IRC (Read error: Operation timed out) [23:07] *** balrog has joined #archiveteam-bs [23:12] *** kyan has joined #archiveteam-bs [23:24] *** bauruine has joined #archiveteam-bs [23:50] *** BlueMaxim has quit IRC (Quit: Leaving) [23:51] *** Apathy has quit IRC (Read error: Operation timed out) [23:51] *** Apathy has joined #archiveteam-bs [23:55] *** bwn has quit IRC (Read error: Operation timed out) [23:58] hmm, yeah, sure looks fucky [23:58] i just got out of work school, i'll poke at it in a few hours