#archiveteam 2016-03-10,Thu

↑back Search

Time Nickname Message
00:00 🔗 mafrasi2_ has quit IRC (No Ping reply in 180 seconds.)
00:03 🔗 mafrasi2 has joined #archiveteam
00:04 🔗 mhazinsk has joined #archiveteam
00:05 🔗 rossdylan has joined #archiveteam
00:07 🔗 aMunster has joined #archiveteam
00:15 🔗 Famicoma1 has quit IRC (Read error: Connection reset by peer)
00:17 🔗 Famicoma1 has joined #archiveteam
00:17 🔗 nwf has joined #archiveteam
00:36 🔗 closure has joined #archiveteam
00:55 🔗 MMovie has joined #archiveteam
01:01 🔗 Jonimus has joined #archiveteam
01:12 🔗 fie has joined #archiveteam
01:12 🔗 JesseW has joined #archiveteam
01:15 🔗 fie_ has quit IRC (Ping timeout: 258 seconds)
01:16 🔗 pikhq has joined #archiveteam
01:19 🔗 JesseW has quit IRC (Quit: Leaving.)
01:29 🔗 Start has joined #archiveteam
02:08 🔗 philpem has quit IRC (Ping timeout: 260 seconds)
02:28 🔗 JesseW has joined #archiveteam
02:46 🔗 Guest__ has quit IRC (Quit: My MacBook Pro has gone to sleep. ZZZzzz…)
02:58 🔗 Zebranky has quit IRC (Read error: Operation timed out)
02:58 🔗 Zebranky has joined #archiveteam
03:22 🔗 Apathy has quit IRC (Read error: Operation timed out)
03:24 🔗 Apathy has joined #archiveteam
03:38 🔗 Ungstein has quit IRC (Quit: Leaving.)
03:41 🔗 w0rp has quit IRC (Ping timeout: 258 seconds)
03:41 🔗 nekomune has quit IRC (Ping timeout: 258 seconds)
03:42 🔗 w0rp has joined #archiveteam
03:42 🔗 pikhq has quit IRC (Ping timeout: 258 seconds)
03:42 🔗 Peetz0r has quit IRC (Ping timeout: 258 seconds)
03:42 🔗 Frogging has quit IRC (Ping timeout: 258 seconds)
03:42 🔗 Peetz0r has joined #archiveteam
03:42 🔗 Frogging has joined #archiveteam
03:43 🔗 GChriss has quit IRC (Ping timeout: 258 seconds)
03:43 🔗 MrRadar has quit IRC (Ping timeout: 258 seconds)
03:43 🔗 chfoo has quit IRC (Ping timeout: 258 seconds)
03:43 🔗 goekesmi has quit IRC (Ping timeout: 258 seconds)
03:43 🔗 espes__ has quit IRC (Ping timeout: 258 seconds)
03:43 🔗 goekesmi has joined #archiveteam
03:43 🔗 chfoo has joined #archiveteam
03:43 🔗 Ungstein has joined #archiveteam
03:44 🔗 espes__ has joined #archiveteam
03:45 🔗 MrRadar has joined #archiveteam
03:45 🔗 GChriss has joined #archiveteam
03:45 🔗 kcaj has quit IRC (Ping timeout: 258 seconds)
03:47 🔗 Lord_Nigh has quit IRC (Ping timeout: 258 seconds)
03:48 🔗 kcaj has joined #archiveteam
03:48 🔗 MrRadar_ has joined #archiveteam
03:49 🔗 MrRadar_ has quit IRC (Client Quit)
03:51 🔗 ersi has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 z00nx has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 will has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 Marc has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 useretail has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 Rye has quit IRC (hub.dk irc.underworld.no)
03:51 🔗 Nemo_bis has quit IRC (hub.dk irc.underworld.no)
03:52 🔗 Lord_Nigh has joined #archiveteam
03:54 🔗 pikhq has joined #archiveteam
03:56 🔗 will__ has joined #archiveteam
03:57 🔗 ersi_ has joined #archiveteam
03:57 🔗 bwn has quit IRC (Read error: Operation timed out)
04:07 🔗 will__ is now known as will
04:32 🔗 wp494 has quit IRC (Read error: Connection reset by peer)
05:08 🔗 Sk1d has quit IRC (Ping timeout: 194 seconds)
05:14 🔗 Sk1d has joined #archiveteam
05:32 🔗 wp494 has joined #archiveteam
05:50 🔗 JesseW has quit IRC (Quit: Leaving.)
06:03 🔗 WinterFox has joined #archiveteam
06:28 🔗 fie_ has joined #archiveteam
06:30 🔗 fie has quit IRC (Read error: Operation timed out)
07:10 🔗 roninski has quit IRC (Read error: Connection reset by peer)
07:11 🔗 roninski has joined #archiveteam
07:20 🔗 ersi_ is now known as ersi
07:23 🔗 roninski has quit IRC (Read error: Connection timed out)
07:24 🔗 roninski has joined #archiveteam
07:32 🔗 RichardG_ has joined #archiveteam
07:32 🔗 RichardG has quit IRC (Ping timeout: 260 seconds)
07:34 🔗 metalcamp has joined #archiveteam
07:38 🔗 bwn has joined #archiveteam
07:40 🔗 RichardG has joined #archiveteam
07:41 🔗 RichardG_ has quit IRC (Ping timeout: 258 seconds)
07:41 🔗 useretail has joined #archiveteam
07:41 🔗 Nemo_bis has joined #archiveteam
07:41 🔗 Rye has joined #archiveteam
07:41 🔗 Marc has joined #archiveteam
07:43 🔗 z00nx has joined #archiveteam
07:45 🔗 nekomune has joined #archiveteam
08:00 🔗 RichardG_ has joined #archiveteam
08:04 🔗 RichardG has quit IRC (Read error: Operation timed out)
08:06 🔗 metal_cam has joined #archiveteam
08:08 🔗 metalcamp has quit IRC (Ping timeout: 250 seconds)
08:17 🔗 vOYtEC has quit IRC (rm -r *)
08:17 🔗 metal_cam is now known as metalcamp
08:19 🔗 ndiddy has quit IRC (Read error: Connection reset by peer)
08:23 🔗 Cameron_D has joined #archiveteam
08:50 🔗 RichardG has joined #archiveteam
08:54 🔗 Guest_ has joined #archiveteam
08:54 🔗 atomotic has joined #archiveteam
08:55 🔗 RichardG_ has quit IRC (Ping timeout: 633 seconds)
08:56 🔗 metalcamp has quit IRC (Ping timeout: 244 seconds)
09:11 🔗 RichardG_ has joined #archiveteam
09:16 🔗 RichardG has quit IRC (Read error: Operation timed out)
09:17 🔗 bwn has quit IRC (Ping timeout: 492 seconds)
09:33 🔗 RichardG has joined #archiveteam
09:34 🔗 brabo has quit IRC (Read error: Operation timed out)
09:35 🔗 RichardG_ has quit IRC (Ping timeout: 272 seconds)
09:38 🔗 bwn has joined #archiveteam
09:42 🔗 roninski1 has joined #archiveteam
09:43 🔗 rossdylan has quit IRC (Read error: Operation timed out)
09:44 🔗 roninski has quit IRC (Ping timeout: 260 seconds)
09:44 🔗 brabo has joined #archiveteam
09:56 🔗 rossdylan has joined #archiveteam
09:56 🔗 jut has joined #archiveteam
10:06 🔗 schbirid has joined #archiveteam
11:04 🔗 SmileyG has joined #archiveteam
11:04 🔗 winr5r has joined #archiveteam
11:05 🔗 winr4r has quit IRC (Read error: Operation timed out)
11:05 🔗 Jogie has quit IRC (Read error: Operation timed out)
11:06 🔗 Mayeau has joined #archiveteam
11:06 🔗 lbft_ has joined #archiveteam
11:06 🔗 Mayonaise has quit IRC (Read error: Operation timed out)
11:08 🔗 roninski has joined #archiveteam
11:08 🔗 roninski1 has quit IRC (Ping timeout: 260 seconds)
11:09 🔗 schbirid has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 Zebranky has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 sivoais has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 Smiley has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 patrickod has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 lbft has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 oli has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 achip has quit IRC (hub.efnet.us irc.Prison.NET)
11:09 🔗 oli_ has joined #archiveteam
11:15 🔗 schbirid2 has joined #archiveteam
11:16 🔗 achip has joined #archiveteam
11:17 🔗 patrickod has joined #archiveteam
11:20 🔗 Zebranky_ has joined #archiveteam
11:24 🔗 oli_ is now known as oli
11:31 🔗 metalcamp has joined #archiveteam
11:38 🔗 sivoais has joined #archiveteam
11:43 🔗 metal_cam has joined #archiveteam
11:45 🔗 metalcamp has quit IRC (Ping timeout: 244 seconds)
11:54 🔗 metal_cam has quit IRC (Ping timeout: 244 seconds)
12:20 🔗 jut has quit IRC (Read error: Connection reset by peer)
12:21 🔗 JSharp___ has joined #archiveteam
12:21 🔗 Ctrl-S___ has joined #archiveteam
12:21 🔗 karissa__ has joined #archiveteam
12:21 🔗 VonGuard has joined #archiveteam
12:32 🔗 VADemon has joined #archiveteam
12:36 🔗 mafrasi2 has quit IRC (Remote host closed the connection)
12:36 🔗 PotcFdk has quit IRC (Ping timeout: 506 seconds)
12:38 🔗 mafrasi2 has joined #archiveteam
12:42 🔗 PotcFdk has joined #archiveteam
12:56 🔗 robink has quit IRC (Ping timeout: 633 seconds)
13:11 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
13:16 🔗 scyther has joined #archiveteam
13:43 🔗 atomotic has joined #archiveteam
13:56 🔗 ete has joined #archiveteam
14:18 🔗 WinterFox has quit IRC (Remote host closed the connection)
14:19 🔗 vitzli has joined #archiveteam
15:10 🔗 wvdp has joined #archiveteam
15:12 🔗 Start has quit IRC (Quit: Disconnected.)
15:13 🔗 RichardG_ has joined #archiveteam
15:13 🔗 RichardG has quit IRC (Ping timeout: 272 seconds)
15:21 🔗 atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
15:44 🔗 pfallenop has quit IRC (Ping timeout: 260 seconds)
15:45 🔗 Start has joined #archiveteam
15:50 🔗 pfallenop has joined #archiveteam
16:02 🔗 Froggypwn has joined #archiveteam
16:03 🔗 JesseW has joined #archiveteam
16:21 🔗 jmad980 has quit IRC (Read error: Operation timed out)
16:24 🔗 jmad980 has joined #archiveteam
16:29 🔗 SketchCow "Is this something the Internet Archive would be interested in? From longtime wnyc dj, David Garland, .I've heard that WNYC Radio will soon remove much of the audio archive for my show Spinning On Air, which I presented there for nearly 30 years. Listen while you can! This link takes you to the archive, where you'll find many episodes, each of which has its own page. Feel free to share this link, o
16:29 🔗 SketchCow r episodes that feature your favorites among my many brilliant guests."
16:30 🔗 arkiver what's the link?
16:31 🔗 SketchCow http://www.wnyc.org/shows/spinning/
16:32 🔗 arkiver sounds like something for godane :)
16:32 🔗 SketchCow http://www.podtrac.com/pts/redirect.mp3/audio.wnyc.org/spinning/spinning091612.mp3
16:32 🔗 arkiver yeah
16:36 🔗 JesseW has quit IRC (Quit: Leaving.)
16:36 🔗 godane starting to download it now
16:36 🔗 arkiver :D
16:36 🔗 arkiver godane: can you also grab everything as WARCsbesides mp3 for the audio items?
16:37 🔗 JesseW has joined #archiveteam
16:37 🔗 arkiver Also, we now have around 400 million URLs of fotolog saved!!!
16:37 🔗 SketchCow Don't I know it.
16:37 🔗 JesseW has quit IRC (Client Quit)
17:05 🔗 Darkstar has joined #archiveteam
17:09 🔗 Start has quit IRC (Quit: Disconnected.)
17:09 🔗 atomotic has joined #archiveteam
17:12 🔗 roninski1 has joined #archiveteam
17:13 🔗 roninski has quit IRC (Ping timeout: 260 seconds)
17:20 🔗 Guest_ Weirdest thing ever. I did this tool for Archive.org and now I'm experiencing like 10k visitors a day, mostly from Canada (?)
17:20 🔗 JW_work what tool?
17:20 🔗 Guest_ JW_work waybackr.com
17:20 🔗 Guest_ is now known as eddiedean
17:20 🔗 JW_work ah, I thought that was you
17:21 🔗 eddiedean Hehe
17:21 🔗 eddiedean I'm trying to figure out if some botnet targeted the website or what.
17:22 🔗 JW_work It's still not very clear to me what it offers. I tried it with a minimal, one page site and it hand that back to me as a zip file — but what does it do for larger things?
17:25 🔗 eddiedean Well, it is send to you via download. I mean, in your email you receive a download link.
17:26 🔗 eddiedean *sent
17:28 🔗 eddiedean Gotta go. I'll be later around, and we can talk :)
17:28 🔗 eddiedean has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
17:29 🔗 Froggypwn has quit IRC (Ping timeout: 1208 seconds)
17:29 🔗 atomotic_ has joined #archiveteam
17:29 🔗 atomotic_ has quit IRC (Client Quit)
17:31 🔗 Tomcat_ has joined #archiveteam
17:31 🔗 atomotic has quit IRC (Ping timeout: 260 seconds)
17:46 🔗 Tomcat_ has quit IRC (Remote host closed the connection)
17:50 🔗 scyther has quit IRC (Quit: Leaving)
17:53 🔗 arkiver Guest_: are you currently doing something with flickr and the wayback machine api>
17:53 🔗 arkiver ?
17:53 🔗 arkiver eddiedean ^
17:54 🔗 RedType has joined #archiveteam
17:54 🔗 arkiver oh, eddiedean is gone
17:55 🔗 jut has joined #archiveteam
17:56 🔗 Froggypwn has joined #archiveteam
17:57 🔗 Ymgve has quit IRC ()
17:58 🔗 vitzli has quit IRC (Leaving)
18:05 🔗 philpem has joined #archiveteam
18:23 🔗 SN4T14 has quit IRC (Remote host closed the connection)
18:26 🔗 yipdw_ is now known as yipdw
18:28 🔗 SN4T14 has joined #archiveteam
18:34 🔗 JW_work has quit IRC (Read error: Operation timed out)
18:35 🔗 vOYtEC has joined #archiveteam
18:35 🔗 schbirid2 has quit IRC (Remote host closed the connection)
18:40 🔗 metalcamp has joined #archiveteam
18:40 🔗 SketchCow So, I am actively and aggressively downloading the Spinning on Air archives.
18:40 🔗 SketchCow Only downside, if you want to call it that, is that it is REALLY a slow download. 100k/s
18:40 🔗 SketchCow So, 8 minutes a podcast. 50mb apiece.
18:41 🔗 JW_work1 has joined #archiveteam
18:42 🔗 JW_work has joined #archiveteam
18:45 🔗 JW_work2 has joined #archiveteam
18:46 🔗 bai would more download streams help, or are they just straining their upstream?
18:46 🔗 godane SketchCow: i'm also downloading them
18:46 🔗 JW_work has quit IRC (Read error: Operation timed out)
18:47 🔗 JW_work1 has quit IRC (Ping timeout: 370 seconds)
18:48 🔗 JW_work has joined #archiveteam
18:48 🔗 JW_work2 has quit IRC (Read error: Operation timed out)
18:50 🔗 Tomcat_ has joined #archiveteam
18:53 🔗 ohhdemgir has joined #archiveteam
18:59 🔗 godane uploaded: https://archive.org/details/wnyc-spinning-on-air-08-14-2005
19:00 🔗 ete has quit IRC (Read error: Operation timed out)
19:02 🔗 LastNinja has joined #archiveteam
19:02 🔗 LastNinja I'm back - been away for a few days and connection dropped out
19:06 🔗 Atluxity welcome back
19:07 🔗 schbirid has joined #archiveteam
19:21 🔗 Start has joined #archiveteam
19:24 🔗 ErkDog WB
19:26 🔗 ErkDog SketchCow I'm getting 2.3MB/sec from that site
19:30 🔗 Ymgve has joined #archiveteam
19:37 🔗 roninski1 has quit IRC (Ping timeout: 260 seconds)
19:39 🔗 roninski has joined #archiveteam
19:44 🔗 SketchCow Woo hoo
19:45 🔗 SketchCow godane: It's good you're grabbing them, but can you ensure all the metadata gets in there?
19:49 🔗 bwn has quit IRC (Ping timeout: 246 seconds)
19:51 🔗 ErkDog yeah I don't know how to do proper manual grabs of things :( I just make the scripts go, lol
19:53 🔗 arkiver godane: and please also do them as WARCs!!!
19:59 🔗 arkiver SketchCow: are you able to increase rsync connections to FOS?
20:00 🔗 arkiver Fotolog is really slowing down due to the 100 connections limit
20:00 🔗 arkiver And they should have shutdown quite some time ago, so I'm not sure how long they'll be up
20:00 🔗 arkiver Or can FOS not handle that at the moment?
20:04 🔗 Tomcat_ has quit IRC (Remote host closed the connection)
20:05 🔗 ErkDog arkiver we've been talking about FOS being slow for days :(
20:05 🔗 ErkDog I stopped my gametracker & wiki grabs, because it's just beating on FOS more
20:05 🔗 ErkDog I stopped them 4 days ago, and they are still trying to push out the pidly 20G that was stacked up on them
20:08 🔗 arkiver You can cancel all that then
20:09 🔗 arkiver I'll just requeue your items
20:09 🔗 arkiver fotolog really is a lot more important
20:09 🔗 ErkDog no I'm letting them flush, there's only 1 left
20:09 🔗 ErkDog it's just the last 1 of them that's finishing
20:09 🔗 phuzion arkiver: are we using FOS as the only rsync target for fotolog?
20:09 🔗 arkiver currently yes
20:09 🔗 ErkDog but from what I've seen and other people are saying, nobody can push more than 100-150KBytes/sec to FOS
20:10 🔗 phuzion arkiver: If we had another rsync box, that could possibly alleviate some of the headache, right?
20:11 🔗 arkiver definitely
20:11 🔗 phuzion Atluxity: Still around?
20:11 🔗 arkiver who owns fboyd?
20:11 🔗 HCross Fletcher
20:11 🔗 arkiver ok
20:11 🔗 arkiver Fletcher: I still have your target in the fotolog grab (deactivated)
20:12 🔗 arkiver Can we activate that for the WARC parts of the fotolog grab?
20:12 🔗 kris33 has joined #archiveteam
20:13 🔗 Microguru has joined #archiveteam
20:18 🔗 ErkDog I'd spin up fotolog right now on 2 systems, but without an RSync target besides FOS, I'm not really doing anything usefull, and just slowing everyone else down
20:19 🔗 Atluxity phuzion: hanging in here...
20:19 🔗 ErkDog if everyone that's here, could stop everything but fotolog that would help
20:19 🔗 HCross ErkDog, are they long term instances?
20:19 🔗 ErkDog beacuse then only fotolog would be going to FOS
20:19 🔗 ErkDog yes I can run them indefinitely
20:19 🔗 kris33 has quit IRC (Textual IRC Client: www.textualapp.com)
20:19 🔗 bwn has joined #archiveteam
20:19 🔗 HCross hop in #newsgrabber
20:19 🔗 HCross we can put them to use
20:19 🔗 phuzion Atluxity: You hosted an rsync target at one point, didn't you?
20:19 🔗 ErkDog http://tracker.archiveteam.org/ftp/
20:19 🔗 ErkDog I can push and pull some damn packets
20:19 🔗 ErkDog just have to have somewhere to push them
20:20 🔗 ErkDog I've done 2.1TB of FTP in less than a week bros
20:20 🔗 Atluxity phuzion: no, I got a host ready to do so, but never got around to set up the rsync part
20:20 🔗 HCross ErkDog, newsgrabber doesnt use FOS
20:20 🔗 ErkDog lol you are trying to solicit me for newsgrabber ;)
20:20 🔗 HCross yes
20:20 🔗 Atluxity phuzion: also, I am unsure about the speed available, that can be... interesting
20:20 🔗 ErkDog is there an e-Penis I mean tracker for it yet?
20:20 🔗 HCross not yet, its done differently
20:20 🔗 ErkDog awwwwwwww
20:20 🔗 phuzion Atluxity: How much disk do you have available?
20:21 🔗 HCross arkiver, we need an "e-Penis tracker"
20:21 🔗 Atluxity phuzion: 10TB
20:21 🔗 phuzion HCross: Leaderboards need to be represented by ascii dongs. 8=======> longer the dong, more data you've transferred.
20:21 🔗 ErkDog SWEEEEEEEEEEEEEEEEEEEEET
20:21 🔗 * ErkDog agrees
20:23 🔗 phuzion Atluxity: Are you willing to use that 10TB as an rsync target? I could possibly help you out a little bit with the setup
20:24 🔗 Atluxity phuzion: I would be glad to give you access to it to help me out
20:25 🔗 Atluxity I am not quite sure how well monitored the raid is, now that I come to think of why I did not go ahead last time I had time
20:25 🔗 Atluxity :\
20:25 🔗 phuzion Awesome. I don't know what needs done for the megawarcing and whatnot, that's something that we'd need someone else's input on
20:25 🔗 phuzion Oh
20:30 🔗 Atluxity phuzion: I got word the raid is monitored
20:31 🔗 phuzion ok
20:31 🔗 ErkDog add him as target for wiki & gametracker & fotolog
20:31 🔗 ErkDog so FOS can breath, lol
20:31 🔗 Atluxity eeeeek
20:31 🔗 MrRadar It might be better to dedicate Atluxity's server to fotolog since that's most at risk right now
20:32 🔗 phuzion That's what I was thinking.
20:32 🔗 phuzion (MrRadar's idea)
20:35 🔗 Atluxity so use the new untested thingy to handle the most endagered stuff?
20:35 🔗 RedType has quit IRC (Read error: Operation timed out)
20:35 🔗 Atluxity how bad can it be?
20:36 🔗 Atluxity whats the worst that could happen...
20:36 🔗 chfoo has quit IRC (Read error: Operation timed out)
20:36 🔗 ersi has quit IRC (Read error: Operation timed out)
20:36 🔗 Smiley has joined #archiveteam
20:36 🔗 Atluxity phuzion: ssh pub key to me privmsg, and I can give you access
20:36 🔗 Jonimus has quit IRC (Read error: Operation timed out)
20:36 🔗 SmileyG has quit IRC (Read error: Operation timed out)
20:36 🔗 Atluxity I need to go to bed soon
20:36 🔗 aMunster has quit IRC (Read error: Operation timed out)
20:36 🔗 rossdylan has quit IRC (Write error: Broken pipe)
20:36 🔗 beardicus has quit IRC (Read error: Operation timed out)
20:36 🔗 nwf has quit IRC (Read error: Operation timed out)
20:36 🔗 sivoais has quit IRC (Read error: Operation timed out)
20:37 🔗 chfoo has joined #archiveteam
20:37 🔗 sivoais has joined #archiveteam
20:37 🔗 MMovie has quit IRC (Read error: Operation timed out)
20:37 🔗 lbft_ has quit IRC (Read error: Operation timed out)
20:37 🔗 Mayeau has quit IRC (Read error: Operation timed out)
20:38 🔗 vegbrasil has quit IRC (Read error: Operation timed out)
20:38 🔗 oli has quit IRC (Read error: Operation timed out)
20:39 🔗 lbft has joined #archiveteam
20:39 🔗 oli has joined #archiveteam
20:41 🔗 ErkDog LOL Atluxity :-D
20:42 🔗 closure has quit IRC (Read error: Operation timed out)
20:42 🔗 ersi has joined #archiveteam
20:43 🔗 Asparagir has joined #archiveteam
20:44 🔗 Start has quit IRC (Quit: Disconnected.)
20:46 🔗 Atluxity ErkDog :)
20:46 🔗 ErkDog I guess having it and maybe something going wrong is better than not having it at all because FOS is overloaded ;(
20:46 🔗 VADemon has quit IRC (Quit: left4dead)
20:47 🔗 Atluxity ofc
20:47 🔗 mhazinsk has quit IRC (Ping timeout: 633 seconds)
20:47 🔗 Atluxity but this can look a bit like a ddos
20:47 🔗 Atluxity and if the speed is any throttled, which is untested, then it will have the same effect
20:48 🔗 RedType has joined #archiveteam
20:50 🔗 Atluxity hmm... someone sais the uplink is 100mbit :\
20:50 🔗 Atluxity thats residential speeds
20:51 🔗 Frogging no
20:52 🔗 ErkDog well -decent- firewalls would see that it's legitimate traffic, I wouldn't say transfers from 20-30 locations would be a "ddos" per se
20:56 🔗 Atluxity 20-30 locations? I run 50 myself
20:57 🔗 ErkDog well that's probably why FOS is shitting all over itself
20:57 🔗 yipdw ErkDog is what happens when you anthropomorphize pv(1)
20:57 🔗 ete has joined #archiveteam
20:57 🔗 ErkDog if ONE person is pushing 50 rsynch's to it
20:58 🔗 ErkDog like do you know how harddrives work? and storage systems? lol
20:58 🔗 xmc yipdw: ha
20:58 🔗 xmc Atluxity: is it you who's aiming the proton cannon at fos?
20:58 🔗 Atluxity ErkDog: if we want less pipelines, we can throttle the tracker?
20:59 🔗 ErkDog well true, but the problem is, I don't think anybody has any real way of seeing how many incoming rsynch's there are
20:59 🔗 Atluxity xmc: I think I might be 50% of the reason...
20:59 🔗 ErkDog and whether it 'needed' to be throttled
20:59 🔗 ErkDog and since there are half a dozen different trackers sending jobs in
20:59 🔗 yipdw I know how many there are, it's not really relevant though if fos needs work
21:00 🔗 Atluxity I trust ppl to let me know when to hold back
21:00 🔗 ErkDog well the fact remains, that any given system can only take X number of RSynch's before you end up with 1/Y performance per Rsync, where Y is a number less than 1
21:00 🔗 ErkDog well Atluxity we've been saying for days how FOS is getting pounded in the face
21:01 🔗 yipdw I wish I could increase performance with more rsync processes
21:01 🔗 yipdw that'd be fucking awesome
21:01 🔗 ErkDog I mean greater than one
21:01 🔗 ErkDog Atluxity assuming no bottlenecks on bandwidth, so say, a 'lan' connection, 1 Rsynch will finish 50 units of work, faster than 50 rsync's will finish 1 unit each
21:02 🔗 Atluxity huh
21:02 🔗 yipdw anyway, fotolog is already at 60 requests/minute, I can push it down a little more and we'll see if it makes any difference
21:02 🔗 arkiver Hmm
21:02 🔗 arkiver It's not big items
21:02 🔗 yipdw I suspect the answer will remain "fos needs work" and we will know when it's fixed because SketchCow will tell us it's fixed
21:02 🔗 arkiver FTP is not going to FOS
21:02 🔗 ErkDog I know FTP isn't
21:02 🔗 arkiver GameTrailers can be paused.
21:02 🔗 ErkDog we are talking about fotolog, wiki & gametrailers
21:03 🔗 arkiver But fotolog is important, do not slow it down, slow other things doen
21:03 🔗 arkiver down*
21:03 🔗 ErkDog specifically targets to Fos
21:03 🔗 arkiver GameFront is paused
21:03 🔗 yipdw there isn't much else to slow down, really
21:03 🔗 ErkDog Atluxity if you have 50 workers going, then you are sending the haddrive to 50 different places all at the same time, whereas if you had 1 worker going it would send one stream of data
21:03 🔗 yipdw rsync traffic to fos is fotolog and gamefront
21:03 🔗 ErkDog and Wiki
21:03 🔗 xmc ErkDog: calm down
21:03 🔗 ErkDog wiki goes to FOS
21:04 🔗 arkiver And wiki is just really small
21:04 🔗 Atluxity ErkDog: I got 50 hosts, running with a concurrency of 10 each...
21:04 🔗 yipdw root@teamarchive0:~# lsof 2>&1 | grep rsync | grep warrior | grep wiki
21:04 🔗 yipdw rsync 5063 1001 cwd DIR 9,1 12288 77596257 /1/CHFOO/warrior/wikis/Floxit
21:04 🔗 yipdw rsync 5077 1001 cwd DIR 9,1 12288 77596257 /1/CHFOO/warrior/wikis/Floxit
21:04 🔗 yipdw rsync 5077 1001 5u REG 9,1 543424512 77594660 /1/CHFOO/warrior/wikis/Floxit/.wikis-mediawikieu_hu.assassinscreed.wikia.com_api.php_hu.assassinscreed.wikia.com_wiki_-20160310-163250.warc.gz.rsZfI1
21:04 🔗 arkiver Like really, not a lot is going to FOS
21:04 🔗 ErkDog well Atluxity said he didn't really understand what I meant
21:04 🔗 yipdw seriously stop telling me what I can find
21:04 🔗 arkiver it's not warrior projects I think
21:04 🔗 yipdw kthxbai
21:04 🔗 yipdw it's fotolog and if we can't slow it down then we'll just wait for fos to spring back
21:04 🔗 yipdw this isn't fatal
21:05 🔗 arkiver Then we'll wait for FOS to spring back
21:05 🔗 yipdw we might also be able to use Kenshin's stuff but I think he is still rebuilding the disk array on that one
21:05 🔗 ErkDog all I was trying to explain to Atluxity since he's running so many hosts, that if he's downloading faster than he can upload to FOS then having more hosts results in diminishing returns, as rsync to an overloaded target becomes the bottleneck
21:05 🔗 xmc YES WE KNOW
21:06 🔗 xmc jfc
21:06 🔗 ErkDog well he didn't seem to know, because when I explained it he said "huh"
21:06 🔗 phuzion ErkDog: I hate to be a debbie-downer here, but you are not telling us anything we don't know.
21:06 🔗 ErkDog and "I trust people to ask me to slow down"
21:06 🔗 phuzion we have literally known for __MONTHS__ that FOS is underpowered.
21:06 🔗 ErkDog was kicked by xmc (learn to keep your mouth shut a bit more)
21:06 🔗 vegbrasil has joined #archiveteam
21:06 🔗 ErkDog has joined #archiveteam
21:06 🔗 yipdw I think that escalated too quickly
21:06 🔗 Asparag-1 has joined #archiveteam
21:06 🔗 arkiver zino__: I'm afraid we need to use your target again
21:07 🔗 arkiver Would you have some rsync space available for fotolog?
21:07 🔗 yipdw zino__ has an rsync target?
21:07 🔗 arkiver Yeah
21:07 🔗 yipdw oh cool
21:07 🔗 zino__ arkiver: Okidokey then. :)
21:07 🔗 Asparag-1 has left
21:07 🔗 arkiver He still holds 12 TB of gametrailers
21:07 🔗 arkiver So we need to move that off still
21:07 🔗 arkiver yipdw: would you be able to move that 12 TB to IA?
21:07 🔗 ErkDog well every time I've mentioned FOS someone tells me it's my problem and to provide traceroutes
21:07 🔗 yipdw I could, I don't have access to itthough
21:08 🔗 xmc sets mode: +o yipdw
21:08 🔗 xmc sets mode: +oo chfoo- chfoo
21:08 🔗 xmc sets mode: +o swebb
21:08 🔗 swebb sets mode: +o edsu
21:08 🔗 swebb sets mode: +o ersi
21:08 🔗 ErkDog and I go 'but everyone can only push 100KBps' and then nobody says anything else about it, lol
21:08 🔗 xmc ErkDog: we've heard you loud and clear
21:08 🔗 yipdw if zino__ has space then it'll help a lot of things
21:09 🔗 ErkDog fair enough, also, to be fair, I still don't know many of you or what your roles in this team are.... the only thing I've been able to skry so far is that SketchCow works for archive.org and is in charge of FOS
21:09 🔗 yipdw loudest voice wins
21:09 🔗 ErkDog LOL
21:09 🔗 zino__ Status page says I have 14T left, but still expecting .8T from the ftpgrab. See http://eldrimner.lysator.liu.se:8080/archiveteam.txt
21:09 🔗 xmc loudest voice wins, not everyone gets +o
21:09 🔗 ErkDog ohhhh zino__ is eldrimner you?
21:10 🔗 zino__ Yep
21:10 🔗 ErkDog I've been sending arkiver all kinds of queue's to add :-D
21:10 🔗 johtso Does anyone know of a good way to make sure a CD I'm archiving can be faithfully copied with dd as an ISO or whether I need to copy it as a BIN+CUE?
21:11 🔗 xmc johtso: is it multisession or does it have more than one track => bin/cue
21:11 🔗 arkiver zino__: let's get the gametrailers stuff off soon. I hope yipdw is able to do that
21:11 🔗 arkiver I see betadump and friendsreunited too
21:11 🔗 yipdw arkiver: if you have shell access, you can run the megawarc factory also
21:11 🔗 johtso xmc: is there a simple way to see what tracks a cd-rom has?
21:11 🔗 yipdw or zino__ can
21:11 🔗 arkiver yipdw: ok, will do
21:12 🔗 ErkDog If someone points fotolog @ eldrimner let me know, I'll spin up instances
21:12 🔗 arkiver I never used it though, so will need to figure some things out first
21:12 🔗 yipdw just make sure your S3 keys are granted upload access to that collection
21:12 🔗 arkiver ok
21:12 🔗 xmc johtso: i don't have a good way, sorry
21:13 🔗 zino__ arkiver: betadump (one of the empty ones) was created in case SketchCow needed it. Just in case.
21:13 🔗 arkiver I see
21:13 🔗 arkiver so fotolog is ready?
21:14 🔗 arkiver (I see the dir there)
21:14 🔗 zino__ Fotolog never got directed towards me, so also empty. :)
21:14 🔗 zino__ The ones that show 4.0K have never been used.
21:14 🔗 phuzion So, who here knows exactly what needs done to get an rsync target running?
21:15 🔗 phuzion Megawarcing and whatnot.
21:15 🔗 * yipdw does
21:15 🔗 beardicus has joined #archiveteam
21:15 🔗 yipdw the README in archiveteam-megawarc-factory is pretty good
21:16 🔗 yipdw alard as usual was very meticulous
21:16 🔗 arkiver yep
21:17 🔗 zino__ arkiver: Misread you there I think, yes, fotolog is ready to receive if you need it. rsync module "fotolog".
21:17 🔗 zino__ I set it up last time there was some throuble and never removed it.
21:17 🔗 arkiver ok
21:17 🔗 arkiver I also PMed you about it
21:17 🔗 arkiver but will activate it now
21:18 🔗 MMovie has joined #archiveteam
21:18 🔗 zino__ Yea, but client isn't really good at showing me when I get PMs...
21:19 🔗 arkiver right, I'll let you know in this chat next time
21:19 🔗 arkiver FOS is deactivated for fotolog!
21:21 🔗 schbirid has quit IRC (Quit: Leaving)
21:23 🔗 zino__ ErkDog: Pointing done. Do your thing. :)
21:25 🔗 ErkDog thanks :-D
21:30 🔗 arkiver It looks like the website is a lot faster then it was a week ago
21:42 🔗 jut has quit IRC (Quit: jut)
21:44 🔗 ErkDog bunch of stuff probably pushed up into cloudflare now, lol
21:44 🔗 ErkDog zino__ what's your HDD setup like for that 28T array?
21:45 🔗 Ymgve has quit IRC ()
21:47 🔗 zino__ ErkDog: 12xSpinning rust + LSI Megaraid. Probably close to 5 years old by now.
21:47 🔗 Ymgve has joined #archiveteam
21:48 🔗 arkiver chfoo: can you send me the logs for fotolog? Especially the log from today
21:48 🔗 arkiver Turns out fotolog gives an 'over capacity' message as 200
21:49 🔗 zino__ ErkDog: Dell R515.
21:49 🔗 zino__ arkiver: Well, that's not nice. :-/
21:49 🔗 arkiver yeah, fixing that in the script now
21:50 🔗 ErkDog spinning rust, lol
21:51 🔗 ErkDog Raid 5 or 1+0?
21:51 🔗 godane looks like wnyc.org is down or not displaying pages
21:51 🔗 closure has joined #archiveteam
21:52 🔗 ErkDog http://puu.sh/nBZNs/31c9709137.png
21:52 🔗 ErkDog loads for me
21:53 🔗 zino__ ErkDog: Raid 5+0 if I remember correctly.
21:53 🔗 ErkDog 5+0????'
21:53 🔗 ErkDog ohhhhhhhhhhh
21:54 🔗 ErkDog that's an interesting model but also explains the performance
21:54 🔗 zino__ You mean the slowness? :)
21:54 🔗 ErkDog well no because the 0 part gives you more speed across the individual blocks
21:55 🔗 xmc woop woop woop off-topic siren
21:55 🔗 xmc take it to #archiveteam-bs
21:55 🔗 xmc this is supposed to be a low traffic channel
22:04 🔗 MMovie has quit IRC (Read error: Operation timed out)
22:04 🔗 sigkell has quit IRC (Ping timeout: 260 seconds)
22:05 🔗 aMunster has joined #archiveteam
22:06 🔗 MMovie has joined #archiveteam
22:07 🔗 Jonimus has joined #archiveteam
22:07 🔗 swebb sets mode: +o Jonimus
22:09 🔗 ersi as well as a low-derp channel
22:14 🔗 sigkell has joined #archiveteam
22:14 🔗 JW_work has quit IRC (Quit: Leaving.)
22:16 🔗 WinterFox has joined #archiveteam
22:27 🔗 ndiddy has joined #archiveteam
22:27 🔗 HCross Best way of getting a copy of a phpBB forum into the archive?
22:27 🔗 HCross http://www.astra2forum.com is going away on the 14th
22:28 🔗 xmc archivebot with appropriate ignores
22:28 🔗 xmc or, wget-warc-lua with the phpbb script
22:28 🔗 xmc though i think that's kind of fiddly
22:28 🔗 nwf has joined #archiveteam
22:29 🔗 HCross 4 days is a really stupid notice
22:29 🔗 Mayeau has joined #archiveteam
22:30 🔗 Sk1d has quit IRC (Ping timeout: 250 seconds)
22:31 🔗 mhazinsk has joined #archiveteam
22:31 🔗 ErkDog it's a pretty small site
22:37 🔗 HCross it has a link to Lycos on it, thats how old it is
22:47 🔗 atomotic has joined #archiveteam
22:52 🔗 atomotic has quit IRC (Client Quit)
23:11 🔗 plog99 has joined #archiveteam
23:13 🔗 fpoee has quit IRC (Ping timeout: 250 seconds)
23:31 🔗 wvdp has quit IRC (http://www.kiwiirc.com/ - A hand crafted IRC client)
23:36 🔗 Stilett0 has joined #archiveteam
23:37 🔗 JSharp___ has quit IRC (Ping timeout: 260 seconds)
23:37 🔗 JSharp___ has joined #archiveteam
23:38 🔗 Stiletto has quit IRC (Ping timeout: 260 seconds)
23:51 🔗 Stilett0 has quit IRC (Read error: Operation timed out)

irclogger-viewer