[00:05] SketchCow: good news the numbers are really 92601 to 100500 that don't exist [00:06] i figure it was going to do what i did in the 30xxx ids area [00:06] *it did [00:20] So Tigris went offline at about 19:00 UTC. I guess that's it. [00:23] lennier1: I think I can, it'll be 15GB [01:07] *** mtntmnky_ has joined #archiveteam-bs [01:24] huh, there's like 100MB of redundant data, due to multiple clips pointing at the same video URL (I guess two people clipped the stream at the same point and Twitch merges them?) [02:22] *** SmileyG has quit IRC (Remote host closed the connection) [02:22] *** Smiley has joined #archiveteam-bs [02:23] *** ndiddy has quit IRC (Quit: WeeChat 1.4) [02:24] *** ndiddy has joined #archiveteam-bs [02:57] Uh oh [02:57] So my grab of Turiver finished two nights ago and I just reran some errors. [02:57] But now I noticed that apparently the server sometimes returns 404s for existing threads. [02:59] Or perhaps there was a temporary issue on Saturday in the early morning hours. I'll have to investigate further. [03:13] *** robogoat has joined #archiveteam-bs [03:14] *** godane has quit IRC (Read error: Connection reset by peer) [03:23] *** qw3rty__ has joined #archiveteam-bs [03:29] *** godane has joined #archiveteam-bs [03:31] *** qw3rty_ has quit IRC (Read error: Operation timed out) [03:49] *** nicolas17 has quit IRC (Quit: Konversation terminated!) [04:14] *** BlueMaxim has joined #archiveteam-bs [04:14] *** BlueMax has quit IRC (Read error: Connection reset by peer) [05:16] *** apache2_ has joined #archiveteam-bs [05:16] *** apache2 has quit IRC (Write error: Connection reset by peer) [05:26] *** MrRadar_ has joined #archiveteam-bs [05:26] *** MrRadar has quit IRC (Read error: Operation timed out) [05:29] I've started the Winnipeg Free Press thing [06:43] *** godane has quit IRC (Ping timeout: 492 seconds) [07:06] *** ndiddy has quit IRC (se.hub hub.efnet.us) [07:06] *** sivoais has quit IRC (se.hub hub.efnet.us) [07:06] *** Pixi` has quit IRC (se.hub hub.efnet.us) [07:06] *** scorche has quit IRC (se.hub hub.efnet.us) [07:06] *** achip has quit IRC (se.hub hub.efnet.us) [07:06] *** Somebody2 has quit IRC (se.hub hub.efnet.us) [07:06] *** tobbez has quit IRC (se.hub hub.efnet.us) [07:06] *** Datechnom has quit IRC (se.hub hub.efnet.us) [07:06] *** Ryz has quit IRC (se.hub hub.efnet.us) [07:06] *** atomicthu has quit IRC (se.hub hub.efnet.us) [07:06] *** katocala has quit IRC (se.hub hub.efnet.us) [07:06] *** RichardG_ has quit IRC (se.hub hub.efnet.us) [07:06] *** zerkalo has quit IRC (se.hub hub.efnet.us) [07:06] *** TC01 has quit IRC (se.hub hub.efnet.us) [07:06] *** fuzzy8021 has quit IRC (se.hub hub.efnet.us) [07:06] *** girst has quit IRC (se.hub hub.efnet.us) [07:06] *** legoktm has quit IRC (se.hub hub.efnet.us) [07:06] *** antomatic has quit IRC (se.hub hub.efnet.us) [07:06] *** phuzion has quit IRC (se.hub hub.efnet.us) [07:06] *** svchfoo3 has quit IRC (se.hub hub.efnet.us) [07:06] *** Hooloovoo has quit IRC (se.hub hub.efnet.us) [07:06] *** second_ has quit IRC (se.hub hub.efnet.us) [07:06] *** Jonimoose has quit IRC (se.hub hub.efnet.us) [07:06] *** MrRadar_ has quit IRC (se.hub hub.efnet.us) [07:06] *** Stiletto has quit IRC (se.hub hub.efnet.us) [07:06] *** Mayonaise has quit IRC (se.hub hub.efnet.us) [07:06] *** fredgido has quit IRC (se.hub hub.efnet.us) [07:06] *** Jake has quit IRC (se.hub hub.efnet.us) [07:06] *** Wingy has quit IRC (se.hub hub.efnet.us) [07:06] *** wp494 has quit IRC (se.hub hub.efnet.us) [07:06] *** systwi has quit IRC (se.hub hub.efnet.us) [07:06] *** ats has quit IRC (se.hub hub.efnet.us) [07:06] *** Zerote has quit IRC (se.hub hub.efnet.us) [07:06] *** asdf0101 has quit IRC (se.hub hub.efnet.us) [07:06] *** dxrt_ has quit IRC (se.hub hub.efnet.us) [07:06] *** twigfoot has quit IRC (se.hub hub.efnet.us) [07:06] *** paul2520 has quit IRC (se.hub hub.efnet.us) [07:06] *** kisspunch has quit IRC (se.hub hub.efnet.us) [07:06] *** robogoat has quit IRC (se.hub hub.efnet.us) [07:06] *** mtntmnky_ has quit IRC (se.hub hub.efnet.us) [07:06] *** lennier1 has quit IRC (se.hub hub.efnet.us) [07:06] *** ranma has quit IRC (se.hub hub.efnet.us) [07:06] *** DigiDigi has quit IRC (se.hub hub.efnet.us) [07:06] *** jrwr has quit IRC (se.hub hub.efnet.us) [07:06] *** sknebel has quit IRC (se.hub hub.efnet.us) [07:06] *** Igloo has quit IRC (se.hub hub.efnet.us) [07:06] *** SynMonger has quit IRC (se.hub hub.efnet.us) [07:06] *** Larsenv has quit IRC (se.hub hub.efnet.us) [07:06] *** chfoo has quit IRC (se.hub hub.efnet.us) [07:06] *** pie_[bnc] has quit IRC (se.hub hub.efnet.us) [07:06] *** superkuh has quit IRC (se.hub hub.efnet.us) [07:06] *** phirephl- has quit IRC (se.hub hub.efnet.us) [07:06] *** underscor has quit IRC (se.hub hub.efnet.us) [07:06] *** nyany has quit IRC (se.hub hub.efnet.us) [07:06] *** prq has quit IRC (se.hub hub.efnet.us) [07:06] *** svchfoo1 has quit IRC (se.hub hub.efnet.us) [07:06] *** Iglooop1 has quit IRC (se.hub hub.efnet.us) [07:06] *** kiska1825 has quit IRC (se.hub hub.efnet.us) [07:06] *** cm has quit IRC (se.hub hub.efnet.us) [07:06] *** yano has quit IRC (se.hub hub.efnet.us) [07:06] *** Selavi has quit IRC (se.hub hub.efnet.us) [07:06] *** Darkstar has quit IRC (se.hub hub.efnet.us) [07:06] *** pikami has quit IRC (se.hub hub.efnet.us) [07:06] *** Frogging has quit IRC (se.hub hub.efnet.us) [07:06] *** jodizzle has quit IRC (se.hub hub.efnet.us) [07:06] *** _niklas_ has quit IRC (se.hub hub.efnet.us) [07:06] *** JAA has quit IRC (se.hub hub.efnet.us) [07:06] *** simon816 has quit IRC (se.hub hub.efnet.us) [07:06] *** kiska has quit IRC (se.hub hub.efnet.us) [07:06] *** swebb has quit IRC (se.hub hub.efnet.us) [07:07] *** MrRadar_ has joined #archiveteam-bs [07:07] *** robogoat has joined #archiveteam-bs [07:07] *** ndiddy has joined #archiveteam-bs [07:07] *** mtntmnky_ has joined #archiveteam-bs [07:07] *** ranma has joined #archiveteam-bs [07:07] *** lennier1 has joined #archiveteam-bs [07:07] *** Stiletto has joined #archiveteam-bs [07:07] *** Datechnom has joined #archiveteam-bs [07:07] *** Mayonaise has joined #archiveteam-bs [07:07] *** DigiDigi has joined #archiveteam-bs [07:07] *** jrwr has joined #archiveteam-bs [07:07] *** sivoais has joined #archiveteam-bs [07:07] *** kiska1825 has joined #archiveteam-bs [07:07] *** Ryz has joined #archiveteam-bs [07:07] *** Pixi` has joined #archiveteam-bs [07:07] *** atomicthu has joined #archiveteam-bs [07:07] *** katocala has joined #archiveteam-bs [07:07] *** sknebel has joined #archiveteam-bs [07:07] *** Igloo has joined #archiveteam-bs [07:07] *** scorche has joined #archiveteam-bs [07:07] *** fredgido has joined #archiveteam-bs [07:07] *** RichardG_ has joined #archiveteam-bs [07:07] *** SynMonger has joined #archiveteam-bs [07:07] *** Jake has joined #archiveteam-bs [07:07] *** Jonimoose has joined #archiveteam-bs [07:07] *** second_ has joined #archiveteam-bs [07:07] *** Hooloovoo has joined #archiveteam-bs [07:07] *** svchfoo3 has joined #archiveteam-bs [07:07] *** kisspunch has joined #archiveteam-bs [07:07] *** swebb has joined #archiveteam-bs [07:07] *** kiska has joined #archiveteam-bs [07:07] *** simon816 has joined #archiveteam-bs [07:07] *** JAA has joined #archiveteam-bs [07:07] *** _niklas_ has joined #archiveteam-bs [07:07] *** jodizzle has joined #archiveteam-bs [07:07] *** Frogging has joined #archiveteam-bs [07:07] *** phuzion has joined #archiveteam-bs [07:07] *** pikami has joined #archiveteam-bs [07:07] *** svchfoo1 has joined #archiveteam-bs [07:07] *** Iglooop1 has joined #archiveteam-bs [07:07] *** prq has joined #archiveteam-bs [07:07] *** Darkstar has joined #archiveteam-bs [07:07] *** antomatic has joined #archiveteam-bs [07:07] *** legoktm has joined #archiveteam-bs [07:07] *** nyany has joined #archiveteam-bs [07:07] *** ny.us.hub sets mode: +ooo svchfoo3 JAA svchfoo1 [07:07] *** girst has joined #archiveteam-bs [07:07] *** underscor has joined #archiveteam-bs [07:07] *** Selavi has joined #archiveteam-bs [07:07] *** paul2520 has joined #archiveteam-bs [07:07] *** fuzzy8021 has joined #archiveteam-bs [07:07] *** phirephl- has joined #archiveteam-bs [07:07] *** TC01 has joined #archiveteam-bs [07:07] *** superkuh has joined #archiveteam-bs [07:07] *** yano has joined #archiveteam-bs [07:07] *** twigfoot has joined #archiveteam-bs [07:07] *** achip has joined #archiveteam-bs [07:07] *** Somebody2 has joined #archiveteam-bs [07:07] *** dxrt_ has joined #archiveteam-bs [07:07] *** asdf0101 has joined #archiveteam-bs [07:07] *** Zerote has joined #archiveteam-bs [07:07] *** cm has joined #archiveteam-bs [07:07] *** pie_[bnc] has joined #archiveteam-bs [07:07] *** chfoo has joined #archiveteam-bs [07:07] *** zerkalo has joined #archiveteam-bs [07:07] *** ats has joined #archiveteam-bs [07:07] *** systwi has joined #archiveteam-bs [07:07] *** wp494 has joined #archiveteam-bs [07:07] *** Wingy has joined #archiveteam-bs [07:07] *** Larsenv has joined #archiveteam-bs [07:07] *** tobbez has joined #archiveteam-bs [07:07] *** AlsoJAA sets mode: +o JAA [07:08] *** JAA sets mode: +o AlsoJAA [08:04] *** BlueMaxim has quit IRC (Quit: Leaving) [09:23] *** Zerote_ has joined #archiveteam-bs [09:25] *** Zerote__ has joined #archiveteam-bs [09:30] *** Zerote has quit IRC (Read error: Operation timed out) [09:31] *** Zerote has joined #archiveteam-bs [09:34] *** Zerote_ has quit IRC (Ping timeout: 622 seconds) [09:36] *** Zerote__ has quit IRC (Read error: Operation timed out) [10:54] *** HP_Archiv has joined #archiveteam-bs [12:49] *** godane has joined #archiveteam-bs [13:54] *** HP_Archiv has quit IRC (Quit: Leaving) [14:09] *** DigiDigi has quit IRC (Quit: Leaving) [14:33] *** HP_Archiv has joined #archiveteam-bs [14:33] *** HP_Archiv has quit IRC (Client Quit) [14:44] *** vitzli has joined #archiveteam-bs [14:48] *** vitzli has quit IRC (Client Quit) [15:04] *** Datechnom has quit IRC (Read error: Connection reset by peer) [15:05] *** Datechnom has joined #archiveteam-bs [15:16] *** DigiDigi has joined #archiveteam-bs [15:16] *** icedice has joined #archiveteam-bs [15:20] *** Arcorann has quit IRC (Read error: Connection reset by peer) [15:29] *** systwi has quit IRC (Read error: Operation timed out) [15:41] *** systwi has joined #archiveteam-bs [16:06] *** fredgido_ has joined #archiveteam-bs [16:13] *** fredgido has quit IRC (Ping timeout: 622 seconds) [16:46] *** lennier2 has joined #archiveteam-bs [16:52] *** lennier1 has quit IRC (Read error: Operation timed out) [16:52] *** lennier2 is now known as lennier1 [17:25] *** icedice has quit IRC (Leaving) [17:32] *** systwi_ has joined #archiveteam-bs [17:38] *** systwi has quit IRC (Ping timeout: 622 seconds) [18:48] *** nicolas17 has joined #archiveteam-bs [19:13] *** RichardG_ is now known as RichardG [19:36] *** HP_Archiv has joined #archiveteam-bs [19:42] *** HP_Archiv has quit IRC (Quit: Leaving) [20:19] *** HP_Archiv has joined #archiveteam-bs [20:30] *** HP_Archiv has quit IRC (Quit: Leaving) [20:46] *** ephemer0l has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.) [20:52] *** ephemer0l has joined #archiveteam-bs [21:01] Winnipeg Free Press comments is running well [21:02] Had a bug in my queue mechanism early on, and recently encountered a problem where the server sends back the template instead of the rendered page, but otherwise fairly smooth [21:49] *** lennier2 has joined #archiveteam-bs [21:52] *** lennier1 has quit IRC (Ping timeout: 272 seconds) [21:52] OrIdow6: are you archiving into WARCs? [21:52] *** lennier2 is now known as lennier1 [21:53] Thought you were a joaquinit alt there for a second, haha [21:54] Not enough question marks. [21:54] yeah we can almost set autoban on ??? haha [21:54] arkiver: Yes, but as discussed earlier, it's a horrible thing that sends parameters through POST and custom headers, so it wouldn't play back nicely in any case [21:54] ah [21:54] well good anyway [21:54] using JAA's tool? [21:54] No [21:55] Requests with the warcio injection thing (introduced to me by JAA) [21:56] Don't actually know if that's "up to standard" with regard to WBM playback [21:56] if it has all headers and payloads (without edits) then it's up to standards [22:00] I think it should. [22:01] It hooks directly into httplib. [22:09] Yeah, it's low-level enough that I'd assume it would do that (and it's coming from Webrecorder, which I would think, from what little I know about them, would have high standards for it), but I haven't compared it with a network trace or anything like that [22:11] it should be fine [22:11] yeah the guy behind webrecorder has high standard [22:12] though there were some strange problems with warcio initially [22:50] *** jrwr has quit IRC (Ping timeout: 260 seconds) [22:51] *** mgrytbak has quit IRC (Ping timeout: 272 seconds) [22:56] *** revi has quit IRC (Ping timeout: 1230 seconds) [22:57] *** SJon___ has quit IRC (Read error: Operation timed out) [22:58] *** picklefac has quit IRC (Read error: Connection timed out) [22:58] *** Vito` has quit IRC (Read error: Connection timed out) [22:59] *** starlord has quit IRC (Ping timeout: 1230 seconds) [22:59] *** fallenoak has quit IRC (Ping timeout: 1230 seconds) [22:59] *** justcool3 has quit IRC (Ping timeout: 1230 seconds) [23:00] *** t3 has quit IRC (Ping timeout: 1230 seconds) [23:00] *** tchaypo_ has quit IRC (Ping timeout: 1230 seconds) [23:00] *** amelia386 has quit IRC (Ping timeout: 1230 seconds) [23:00] *** diggan has quit IRC (Ping timeout: 1230 seconds) [23:01] *** hook54321 has quit IRC (Ping timeout: 1230 seconds) [23:01] *** jesse-s has quit IRC (Ping timeout: 1230 seconds) [23:01] *** abartov__ has quit IRC (Ping timeout: 1230 seconds) [23:01] *** riking_ has quit IRC (Ping timeout: 1230 seconds) [23:02] *** kyledrake has quit IRC (Read error: Connection timed out) [23:02] *** lenary has quit IRC (Read error: Connection timed out) [23:02] *** JSharp___ has quit IRC (Write error: Connection timed out) [23:02] *** mattl has quit IRC (Write error: Connection timed out) [23:02] *** horkermon has quit IRC (Ping timeout: 1230 seconds) [23:03] *** c0mpass has quit IRC (Ping timeout: 1230 seconds) [23:03] *** tech234a has quit IRC (Ping timeout: 1230 seconds) [23:03] *** xit has quit IRC (Ping timeout: 1230 seconds) [23:03] *** Ctrl-S___ has quit IRC (Read error: Connection timed out) [23:05] *** ThisAsYou has quit IRC (Ping timeout: 1230 seconds) [23:06] *** HCross has quit IRC (Read error: Connection timed out) [23:06] *** pnJay has quit IRC (Read error: Connection timed out) [23:06] *** Kaz has quit IRC (Read error: Connection timed out) [23:06] *** DrasticAc has quit IRC (Read error: Connection timed out) [23:09] *** BlueMax has joined #archiveteam-bs [23:13] *** Arcorann has joined #archiveteam-bs [23:41] What? [23:45] Oh, found it in the logs