[00:06] SketchCow: all of kpfa 2010 mp3s are uploaded [00:13] SketchCow: your also going to be getting some more EGM from 1991 [00:13] feb,may, oct of 1991 just got released [00:15] *** Famicoma1 has quit IRC (Read error: Connection reset by peer) [00:17] *** Famicoma1 has joined #archiveteam-bs [00:28] SketchCow to answer your question, as far as I can tell the bottle neck is FOS, because that's what everyone says is slow. I've got sub 100k/sec from two systems in completely different places, and everyone else says that's all they are getting to. [00:29] both systems are on gbps networks and I can push gpbs to other places all day long [00:36] *** closure has joined #archiveteam-bs [00:36] *** midas sets mode: +o closure [00:37] *** botpie91 has joined #archiveteam-bs [00:40] I'm pushing 4Meg/sec from the two systems for FTP Grabs, and it's in europe [01:12] *** fie has joined #archiveteam-bs [01:12] *** JesseW has joined #archiveteam-bs [01:15] *** fie_ has quit IRC (Ping timeout: 258 seconds) [01:16] *** pikhq has joined #archiveteam-bs [01:19] *** JesseW has quit IRC (Quit: Leaving.) [01:29] *** Start has joined #archiveteam-bs [02:28] *** JesseW has joined #archiveteam-bs [02:58] *** Zebranky has quit IRC (Read error: Operation timed out) [02:58] *** Zebranky has joined #archiveteam-bs [03:22] *** Apathy has quit IRC (Read error: Operation timed out) [03:24] *** Apathy has joined #archiveteam-bs [03:41] *** w0rp has quit IRC (Ping timeout: 258 seconds) [03:42] *** w0rp has joined #archiveteam-bs [03:42] *** pikhq has quit IRC (Ping timeout: 258 seconds) [03:42] *** Frogging has quit IRC (Ping timeout: 258 seconds) [03:42] *** Frogging has joined #archiveteam-bs [03:43] *** MrRadar has quit IRC (Ping timeout: 258 seconds) [03:43] *** chfoo has quit IRC (Ping timeout: 258 seconds) [03:43] *** goekesmi has quit IRC (Ping timeout: 258 seconds) [03:43] *** espes__ has quit IRC (Ping timeout: 258 seconds) [03:43] *** goekesmi has joined #archiveteam-bs [03:43] *** chfoo has joined #archiveteam-bs [03:44] *** espes__ has joined #archiveteam-bs [03:45] *** MrRadar has joined #archiveteam-bs [03:47] *** Lord_Nigh has quit IRC (Ping timeout: 258 seconds) [03:48] *** MrRadar_ has joined #archiveteam-bs [03:49] *** MrRadar_ has quit IRC (Client Quit) [03:51] *** ersi has quit IRC (hub.dk irc.underworld.no) [03:51] *** will has quit IRC (hub.dk irc.underworld.no) [03:51] *** useretail has quit IRC (hub.dk irc.underworld.no) [03:51] *** Rye has quit IRC (hub.dk irc.underworld.no) [03:52] *** Lord_Nigh has joined #archiveteam-bs [03:54] *** pikhq has joined #archiveteam-bs [03:56] *** will__ has joined #archiveteam-bs [03:57] *** ersi_ has joined #archiveteam-bs [03:57] *** bwn has quit IRC (Read error: Operation timed out) [04:07] *** will__ is now known as will [04:32] *** wp494 has quit IRC (Read error: Connection reset by peer) [05:08] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [05:14] *** Sk1d has joined #archiveteam-bs [05:32] *** wp494 has joined #archiveteam-bs [05:50] *** JesseW has quit IRC (Quit: Leaving.) [06:12] I'll bring it up when IA is not in crisis mode. [06:28] *** fie_ has joined #archiveteam-bs [06:30] *** fie has quit IRC (Read error: Operation timed out) [06:48] 100k/sec is more than 0k/sec [07:11] Sounds like we just need a different VM in a different area. [07:11] I can see about that. [07:12] It also might be a sketchy router, etc. [07:12] I had one employee go "reallly?" when finding we were using that thing so much. [07:20] *** ersi_ is now known as ersi [07:20] Oh [07:32] *** RichardG_ has joined #archiveteam-bs [07:32] *** RichardG has quit IRC (Ping timeout: 260 seconds) [07:34] *** metalcamp has joined #archiveteam-bs [07:38] *** bwn has joined #archiveteam-bs [07:40] *** RichardG has joined #archiveteam-bs [07:41] *** RichardG_ has quit IRC (Ping timeout: 258 seconds) [07:41] *** useretail has joined #archiveteam-bs [07:41] *** Rye has joined #archiveteam-bs [08:00] *** RichardG_ has joined #archiveteam-bs [08:04] *** RichardG has quit IRC (Read error: Operation timed out) [08:06] *** metal_cam has joined #archiveteam-bs [08:08] *** metalcamp has quit IRC (Ping timeout: 250 seconds) [08:17] *** metal_cam is now known as metalcamp [08:19] *** ndiddy has quit IRC (Read error: Connection reset by peer) [08:23] *** Cameron_D has joined #archiveteam-bs [08:50] *** RichardG has joined #archiveteam-bs [08:55] *** RichardG_ has quit IRC (Ping timeout: 633 seconds) [08:56] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [09:11] *** RichardG_ has joined #archiveteam-bs [09:16] *** RichardG has quit IRC (Read error: Operation timed out) [09:17] *** bwn has quit IRC (Ping timeout: 492 seconds) [09:33] *** RichardG has joined #archiveteam-bs [09:35] *** RichardG_ has quit IRC (Ping timeout: 272 seconds) [09:38] *** bwn has joined #archiveteam-bs [09:46] SketchCow: as a reaction on FOS? in that case we did good. [09:56] *** jut has joined #archiveteam-bs [10:06] *** schbirid has joined #archiveteam-bs [11:04] *** SmileyG has joined #archiveteam-bs [11:04] *** winr5r has joined #archiveteam-bs [11:05] *** winr4r has quit IRC (Read error: Operation timed out) [11:06] *** Mayeau has joined #archiveteam-bs [11:06] *** lbft_ has joined #archiveteam-bs [11:06] *** Mayonaise has quit IRC (Read error: Operation timed out) [11:09] *** schbirid has quit IRC (hub.efnet.us irc.Prison.NET) [11:09] *** Zebranky has quit IRC (hub.efnet.us irc.Prison.NET) [11:09] *** Smiley has quit IRC (hub.efnet.us irc.Prison.NET) [11:09] *** lbft has quit IRC (hub.efnet.us irc.Prison.NET) [11:09] *** achip has quit IRC (hub.efnet.us irc.Prison.NET) [11:15] *** schbirid2 has joined #archiveteam-bs [11:16] *** achip has joined #archiveteam-bs [11:20] *** Zebranky_ has joined #archiveteam-bs [11:31] *** metalcamp has joined #archiveteam-bs [11:43] *** metal_cam has joined #archiveteam-bs [11:45] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [11:54] *** metal_cam has quit IRC (Ping timeout: 244 seconds) [12:20] *** jut has quit IRC (Read error: Connection reset by peer) [12:21] *** Ctrl-S___ has joined #archiveteam-bs [12:21] *** JSharp___ has joined #archiveteam-bs [12:32] *** VADemon has joined #archiveteam-bs [12:36] *** PotcFdk has quit IRC (Ping timeout: 506 seconds) [12:42] *** PotcFdk has joined #archiveteam-bs [12:56] *** robink has quit IRC (Ping timeout: 633 seconds) [13:56] *** ete has joined #archiveteam-bs [14:06] yes 100K/sec is infinitely more than 0k/sec [14:06] but in the interest of being able to grab sites that might be going offline soon, it's not really sufficient [14:11] * ersi rolls eyes [14:11] True. [14:19] *** vitzli has joined #archiveteam-bs [14:26] ErkDog: do a traceroute to archive.org would you? your path might be congested to the point of breakage [14:46] well then is everyone's patch congested? because nobody gets more than 100 - 150... even Archivebots [14:46] but I will provide both traceroutes [14:47] http://paste.nerds.io/lipebewula.avrasm [14:51] http://paste.nerds.io/atalexuxux.avrasm [14:51] and that's to FOS directly [15:12] *** Start has quit IRC (Quit: Disconnected.) [15:13] *** RichardG_ has joined #archiveteam-bs [15:13] *** RichardG has quit IRC (Ping timeout: 272 seconds) [15:45] *** Start has joined #archiveteam-bs [16:03] *** JesseW has joined #archiveteam-bs [16:36] *** JesseW has quit IRC (Quit: Leaving.) [16:37] *** JesseW has joined #archiveteam-bs [16:37] *** JesseW has quit IRC (Client Quit) [16:55] can anyone find old real media links to shows from wnyc? [16:56] proof there was show in real media for Spinning On Air: https://web.archive.org/web/20090109010730/http://www.wnyc.org/shows/spinning/episodes/2005/01/07 [17:05] *** Darkstar has joined #archiveteam-bs [17:06] It looks like the IA has several .ram playlists archived: https://web.archive.org/web/*/http://www.wnyc.org/stream/ram* [17:07] They all reference rtsp://wnyc.rmod.llnwd.net/a90/o1/ and then the files in the query parameters [17:08] Here's even one for Spinning on Air: https://web.archive.org/web/20071016202235/http://wnyc.org/stream/ram?file1=/spinning/spinning021805hr1.ra&file2=/spinning/spinning021805hr2.ra [17:08] Though the IA says the server with that file is down [17:09] *** Start has quit IRC (Quit: Disconnected.) [17:13] It looks like they are currently distributing audio as mp3s, e.g. http://204.93.180.51/news/news20160310_ny_opera_woman_zambello_2w.mp3?downloadId=56d7826f6fb6ed88_QoqLb5mo_0000000Pwi4 (from http://www.wnyc.org/story/bringing-more-diversity-opera/ ) [17:54] *** RedType has joined #archiveteam-bs [17:55] *** jut has joined #archiveteam-bs [17:58] *** vitzli has quit IRC (Leaving) [18:23] *** SN4T14 has quit IRC (Remote host closed the connection) [18:26] *** yipdw_ is now known as yipdw [18:28] *** SN4T14 has joined #archiveteam-bs [18:34] *** JW_work has quit IRC (Read error: Operation timed out) [18:35] *** schbirid2 has quit IRC (Remote host closed the connection) [18:40] *** metalcamp has joined #archiveteam-bs [18:41] *** JW_work1 has joined #archiveteam-bs [18:42] *** JW_work has joined #archiveteam-bs [18:45] *** JW_work2 has joined #archiveteam-bs [18:46] *** JW_work has quit IRC (Read error: Operation timed out) [18:47] *** JW_work1 has quit IRC (Ping timeout: 370 seconds) [18:48] *** JW_work has joined #archiveteam-bs [18:48] *** JW_work2 has quit IRC (Read error: Operation timed out) [18:53] *** ohhdemgir has joined #archiveteam-bs [19:00] *** ete has quit IRC (Read error: Operation timed out) [19:07] *** schbirid has joined #archiveteam-bs [19:10] anyone wanna help stupid me with iptables? i have a proxmox/openvz setup. i have a container that runs http on 8080. i managed to set iptables so that i can access that just fine from the outside. [19:10] but now i want to enable it to access itself via the externalip on 8080. [19:11] dmesg shows me "Dropped by firewall: IN=venet0 OUT= MAC= SRC=internalcontainerip(192.168...) DST=externalserverip LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=30428 DF PROTO=TCP SPT=40363 DPT=8080 WINDOW=14600 RES=0x00 SYN URGP=0" [19:21] *** Start has joined #archiveteam-bs [19:32] botpie91: tell eddiedean that the ridiculous load on waybackr was due to an Archive Team member who couldn't quit you [19:32] yipdw: I'll pass that on when eddiedean is around. [19:49] *** bwn has quit IRC (Ping timeout: 246 seconds) [20:13] *** Microguru has joined #archiveteam-bs [20:19] *** bwn has joined #archiveteam-bs [20:35] *** RedType has quit IRC (Read error: Operation timed out) [20:36] *** chfoo has quit IRC (Read error: Operation timed out) [20:36] *** kvieta has quit IRC (Read error: Operation timed out) [20:36] *** ersi has quit IRC (Read error: Operation timed out) [20:36] *** Smiley has joined #archiveteam-bs [20:36] *** SmileyG has quit IRC (Read error: Operation timed out) [20:36] *** beardicus has quit IRC (Read error: Operation timed out) [20:37] *** chfoo has joined #archiveteam-bs [20:37] *** lbft_ has quit IRC (Read error: Operation timed out) [20:37] *** Mayeau has quit IRC (Read error: Operation timed out) [20:39] *** lbft has joined #archiveteam-bs [20:42] *** closure has quit IRC (Read error: Operation timed out) [20:42] *** ersi has joined #archiveteam-bs [20:42] *** midas sets mode: +o ersi [20:43] *** Asparagir has joined #archiveteam-bs [20:44] *** Start has quit IRC (Quit: Disconnected.) [20:46] *** VADemon has quit IRC (Quit: left4dead) [20:48] *** RedType has joined #archiveteam-bs [20:49] *** botpie91 has quit IRC (Ping timeout: 633 seconds) [20:54] *** botpie91 has joined #archiveteam-bs [20:57] *** ete has joined #archiveteam-bs [21:00] *** botpie91 has quit IRC (Read error: Operation timed out) [21:06] *** kvieta has joined #archiveteam-bs [21:15] *** beardicus has joined #archiveteam-bs [21:21] *** schbirid has quit IRC (Quit: Leaving) [21:42] *** jut has quit IRC (Quit: jut) [21:51] *** closure has joined #archiveteam-bs [21:51] *** midas sets mode: +o closure [21:55] oooops [21:55] Heh [21:55] so yeah with 1+0 you loose 50% of storage but it's fastest thing you can do with failover [21:56] with 5+0 you never loose more than 33% and possibly less [21:56] so if you've got a 5+0 with 12 disks, you probably have 3 R5 blocks of 4 drives which are then split by the 0 over top [21:57] Yea, it's choosen to be as econimical as possible. [21:57] so instead of writing/reading from ONE BIG Raid 5 array. You essentially get the performance of 3 of the array's in 0 instead of just beating on one big mechanical 5 [21:57] so it's not 3 times as fast because of overhad and blah blah of course [21:58] Yea, trust the hardware RAID controller to bottleneck it somewhere. [21:58] but a 5+0 is still significantly faster than just a 5 [22:00] *** wvdp has joined #archiveteam-bs [22:01] midas: good to hear; also FYI Little Boots has another album out [22:01] if you like LB's work you might also enjoy Grimes' work; I've been looping Art Angels way too much [22:02] ErkDog: All things said I think it's the slowest server I have though. This years generation has 42 disks per machine for the data nodes and 12 SSDs per metadataserver. [22:04] *** sigkell has quit IRC (Ping timeout: 260 seconds) [22:14] *** sigkell has joined #archiveteam-bs [22:14] *** JW_work has quit IRC (Quit: Leaving.) [22:16] yeah SSDs are definitely the shit [22:27] *** ndiddy has joined #archiveteam-bs [22:29] *** Mayeau has joined #archiveteam-bs [22:30] *** Sk1d has quit IRC (Ping timeout: 250 seconds) [23:11] *** plog99 has joined #archiveteam-bs [23:13] *** fpoee has quit IRC (Ping timeout: 250 seconds) [23:24] SketchCow: i'm adding metadata to the Spinning On Air manually [23:24] i don't think i can automate [23:31] *** wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client) [23:36] *** Stilett0 has joined #archiveteam-bs [23:37] *** JSharp___ has quit IRC (Ping timeout: 260 seconds) [23:37] *** JSharp___ has joined #archiveteam-bs [23:38] *** Stiletto has quit IRC (Ping timeout: 260 seconds) [23:51] *** Stilett0 has quit IRC (Read error: Operation timed out)