[00:00] *** kyan has quit IRC (Read error: Connection reset by peer) [00:05] *** grue_ has joined #archiveteam [00:06] *** primus105 has quit IRC (Leaving.) [00:07] *** useretail has joined #archiveteam [00:08] *** Ravenloft has quit IRC (Remote host closed the connection) [00:09] *** primus104 has joined #archiveteam [00:10] *** primus104 has quit IRC (Client Quit) [00:15] *** gui7 has joined #archiveteam [00:19] *** mistym has quit IRC (Quit: Leaving) [00:28] *** grue_ has quit IRC (Quit: leaving) [00:46] *** BlueMaxim has quit IRC (Quit: Leaving) [00:54] *** BlueMaxim has joined #archiveteam [00:59] *** godane has quit IRC (Read error: Operation timed out) [01:06] *** mistym has joined #archiveteam [01:22] *** mistym_ has joined #archiveteam [01:26] *** godane has joined #archiveteam [01:29] *** mistym has quit IRC (Read error: Operation timed out) [01:33] *** Ymgve has quit IRC () [01:45] *** Selanda has joined #archiveteam [02:02] *** [Beta]_ has joined #archiveteam [02:03] *** [Beta] has quit IRC (Ping timeout: 240 seconds) [02:10] *** kyan has joined #archiveteam [02:15] *** mistym_ has quit IRC (Remote host closed the connection) [02:17] *** mistym has joined #archiveteam [02:20] *** yotta has quit IRC (Read error: Operation timed out) [02:20] *** yotta has joined #archiveteam [02:20] *** cloudmons has quit IRC (Read error: Operation timed out) [02:21] *** cloudmons has joined #archiveteam [02:39] *** Ravenloft has joined #archiveteam [02:57] *** SN4T14__ has quit IRC (Read error: Connection reset by peer) [02:57] *** lytv has quit IRC (Read error: Connection reset by peer) [02:58] *** cloudmons has quit IRC (Read error: Operation timed out) [03:00] *** SN4T14 has joined #archiveteam [03:00] *** cloudmons has joined #archiveteam [03:01] *** lytv has joined #archiveteam [03:01] *** chfoo has quit IRC (Read error: Connection reset by peer) [03:03] *** Ravenloft has quit IRC (Ping timeout: 606 seconds) [03:14] *** beardicus has quit IRC (Quit: Sleep.) [03:16] *** chfoo has joined #archiveteam [03:24] *** mistym has quit IRC (Remote host closed the connection) [04:05] *** mistym has joined #archiveteam [04:05] *** aaaaaaaaa has quit IRC (Leaving) [04:07] *** rduser has quit IRC (Read error: Operation timed out) [04:08] *** berndj has quit IRC (Read error: Operation timed out) [04:08] *** rduser has joined #archiveteam [04:11] *** [Beta]_ has quit IRC (Ping timeout: 240 seconds) [04:35] *** mistym has quit IRC (Remote host closed the connection) [05:02] *** mistym has joined #archiveteam [05:45] *** mistym_ has joined #archiveteam [05:46] *** Froggypwn has quit IRC (Read error: Connection reset by peer) [05:47] *** Froggypwn has joined #archiveteam [05:48] *** mistym has quit IRC (Read error: Operation timed out) [06:04] *** philpem has quit IRC (Ping timeout: 260 seconds) [06:09] *** yotta has quit IRC (Read error: Operation timed out) [06:09] *** dashcloud has quit IRC (Read error: Operation timed out) [06:10] *** cloudmons has quit IRC (Read error: Operation timed out) [06:10] *** chfoo has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Selanda has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Jogie has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** signius has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** mutoso has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** bmcginty has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** MMovie2 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dugo has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** espes___ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Mayonaise has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Quile has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** SketchCow has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** ben__ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** closure has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** BlueMaxim has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** thechip has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Fusl has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** xmc has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** phillipsj has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** marnold has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** ersi has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** kanzure_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** okeuday has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** tephra_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** serapeum has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** ryan_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** slash` has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Cameron_D has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dan_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** fx_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** gibigiana has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** balrog has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** marvinw has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** GLaDOS has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dx has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Lord_Nigh has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** kniffy has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Rickster has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** aliz has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Muad-Dib has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** fresco___ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** pfallenop has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** hive-mind has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Stary2001 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** VonGuard has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** russss_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** lhobas has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** mrfoo has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** LittUp has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** johtso has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** mietek has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** deathy has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** lytv has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** godane has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** w0rp has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** filippo_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** antomatic has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Kazzy has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** PepsiMax has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Sk1d has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Deewiant has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** the_fox has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** edsu_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Rotab has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** nico_32 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** brayden has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** NovaKing_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** cadbury_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** warthurto has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Sellyme has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** aNthraXx has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** chfoo- has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Famicoma1 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Sue_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Infreq has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** caber has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Jon has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** garyrh has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** i0npulse has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** altlabel has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Froggypwn has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** gui7 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** raylee has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** svchfoo2 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Atluxity has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** nox has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** jk[SVP] has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** kyan has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** lbft has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** db48x has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** RichardG has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Morbus has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** midas has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** lukeman_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** patrickod has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Zebranky has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** zenguy_pc has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Kenshin has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** torvik has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** wyatt8750 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Coderjoe has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** thefinn93 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** trs80 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** winr4r has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** mhazinsk has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** RedType_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Smiley has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** oli has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Peetz0r has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** eprillios has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** goekesmi has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** swebb has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** fenn has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dinomite has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** SN4T14 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Stiletto has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** ats has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** T31M has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** khaoohs has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Baljem has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** arkiver has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** joepie91 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Meeh has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** anomie has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** yipdw has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dcmorton has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** dserodio has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** svchfoo1 has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** sivoais_ has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** mahadri has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** chazchaz has quit IRC (ircd.choopa.net hub.efnet.us) [06:10] *** Laverne has quit IRC (ircd.choopa.net hub.efnet.us) [06:12] *** cloudmons has joined #archiveteam [06:22] *** dashcloud has joined #archiveteam [06:35] *** berndj has joined #archiveteam [06:35] *** chfoo has joined #archiveteam [06:35] *** lytv has joined #archiveteam [06:35] *** SN4T14 has joined #archiveteam [06:35] *** kyan has joined #archiveteam [06:35] *** Selanda has joined #archiveteam [06:35] *** godane has joined #archiveteam [06:35] *** BlueMaxim has joined #archiveteam [06:35] *** gui7 has joined #archiveteam [06:35] *** Coderjoe has joined #archiveteam [06:35] *** Stiletto has joined #archiveteam [06:35] *** garyrh has joined #archiveteam [06:35] *** thechip has joined #archiveteam [06:35] *** Jogie has joined #archiveteam [06:35] *** ats has joined #archiveteam [06:35] *** T31M has joined #archiveteam [06:35] *** dan_ has joined #archiveteam [06:35] *** khaoohs has joined #archiveteam [06:35] *** signius has joined #archiveteam [06:35] *** raylee has joined #archiveteam [06:35] *** svchfoo2 has joined #archiveteam [06:35] *** fx_ has joined #archiveteam [06:35] *** Atluxity has joined #archiveteam [06:35] *** nox has joined #archiveteam [06:35] *** RedType_ has joined #archiveteam [06:35] *** gibigiana has joined #archiveteam [06:35] *** Fusl has joined #archiveteam [06:35] *** nico_32 has joined #archiveteam [06:35] *** the_fox has joined #archiveteam [06:35] *** mutoso has joined #archiveteam [06:35] *** balrog has joined #archiveteam [06:35] *** lbft has joined #archiveteam [06:35] *** xmc has joined #archiveteam [06:35] *** db48x has joined #archiveteam [06:35] *** i0npulse has joined #archiveteam [06:35] *** brayden has joined #archiveteam [06:35] *** phillipsj has joined #archiveteam [06:35] *** NovaKing_ has joined #archiveteam [06:35] *** RichardG has joined #archiveteam [06:35] *** cadbury_ has joined #archiveteam [06:35] *** marvinw has joined #archiveteam [06:35] *** Baljem has joined #archiveteam [06:35] *** warthurto has joined #archiveteam [06:35] *** Morbus has joined #archiveteam [06:35] *** GLaDOS has joined #archiveteam [06:35] *** dx has joined #archiveteam [06:35] *** ny.us.hub sets mode: +oo balrog xmc [06:35] *** Sellyme has joined #archiveteam [06:35] *** marnold has joined #archiveteam [06:35] *** arkiver has joined #archiveteam [06:35] *** joepie91 has joined #archiveteam [06:35] *** Lord_Nigh has joined #archiveteam [06:35] *** w0rp has joined #archiveteam [06:35] *** aNthraXx has joined #archiveteam [06:35] *** mhazinsk has joined #archiveteam [06:35] *** Laverne has joined #archiveteam [06:35] *** trs80 has joined #archiveteam [06:35] *** winr4r has joined #archiveteam [06:35] *** chazchaz has joined #archiveteam [06:35] *** sivoais_ has joined #archiveteam [06:35] *** fenn has joined #archiveteam [06:35] *** swebb has joined #archiveteam [06:35] *** svchfoo1 has joined #archiveteam [06:35] *** dserodio has joined #archiveteam [06:35] *** goekesmi has joined #archiveteam [06:35] *** dcmorton has joined #archiveteam [06:35] *** eprillios has joined #archiveteam [06:35] *** Peetz0r has joined #archiveteam [06:35] *** mahadri has joined #archiveteam [06:35] *** zenguy_pc has joined #archiveteam [06:35] *** Kenshin has joined #archiveteam [06:35] *** torvik has joined #archiveteam [06:35] *** oli has joined #archiveteam [06:35] *** wyatt8750 has joined #archiveteam [06:35] *** thefinn93 has joined #archiveteam [06:35] *** yipdw has joined #archiveteam [06:35] *** Smiley has joined #archiveteam [06:35] *** anomie has joined #archiveteam [06:35] *** Zebranky has joined #archiveteam [06:35] *** Meeh has joined #archiveteam [06:35] *** patrickod has joined #archiveteam [06:35] *** lukeman_ has joined #archiveteam [06:35] *** midas has joined #archiveteam [06:35] *** dinomite has joined #archiveteam [06:35] *** chfoo- has joined #archiveteam [06:35] *** kniffy has joined #archiveteam [06:35] *** ersi has joined #archiveteam [06:35] *** Famicoma1 has joined #archiveteam [06:35] *** filippo_ has joined #archiveteam [06:35] *** Rickster has joined #archiveteam [06:35] *** Muad-Dib has joined #archiveteam [06:35] *** edsu_ has joined #archiveteam [06:35] *** aliz has joined #archiveteam [06:35] *** ny.us.hub sets mode: +oo Lord_Nigh swebb [06:35] *** mietek has joined #archiveteam [06:35] *** deathy has joined #archiveteam [06:35] *** johtso has joined #archiveteam [06:35] *** mrfoo has joined #archiveteam [06:35] *** lhobas has joined #archiveteam [06:35] *** russss_ has joined #archiveteam [06:35] *** VonGuard has joined #archiveteam [06:35] *** LittUp has joined #archiveteam [06:35] *** Stary2001 has joined #archiveteam [06:35] *** hive-mind has joined #archiveteam [06:35] *** pfallenop has joined #archiveteam [06:35] *** fresco___ has joined #archiveteam [06:35] *** kanzure_ has joined #archiveteam [06:35] *** Sue_ has joined #archiveteam [06:35] *** okeuday has joined #archiveteam [06:35] *** tephra_ has joined #archiveteam [06:35] *** jk[SVP] has joined #archiveteam [06:35] *** serapeum has joined #archiveteam [06:35] *** bmcginty has joined #archiveteam [06:35] *** MMovie2 has joined #archiveteam [06:35] *** dugo has joined #archiveteam [06:35] *** antomatic has joined #archiveteam [06:35] *** Infreq has joined #archiveteam [06:35] *** ryan_ has joined #archiveteam [06:35] *** Cameron_D has joined #archiveteam [06:35] *** slash` has joined #archiveteam [06:35] *** espes___ has joined #archiveteam [06:35] *** closure has joined #archiveteam [06:35] *** ben__ has joined #archiveteam [06:35] *** SketchCow has joined #archiveteam [06:35] *** Quile has joined #archiveteam [06:35] *** Mayonaise has joined #archiveteam [06:35] *** Deewiant has joined #archiveteam [06:35] *** caber has joined #archiveteam [06:35] *** Rotab has joined #archiveteam [06:35] *** Kazzy has joined #archiveteam [06:35] *** PepsiMax has joined #archiveteam [06:35] *** Sk1d has joined #archiveteam [06:35] *** Jon has joined #archiveteam [06:35] *** altlabel has joined #archiveteam [06:35] *** ny.us.hub sets mode: +ooo Cameron_D closure SketchCow [06:35] *** swebb sets mode: +o underscor [06:35] *** swebb sets mode: +o DFJustin [06:35] *** swebb sets mode: +o SadDM [06:45] *** appledash has joined #archiveteam [06:55] *** primus104 has joined #archiveteam [06:55] *** phillipsj has quit IRC (Ping timeout: 512 seconds) [06:56] *** mistym_ has quit IRC (Remote host closed the connection) [06:58] *** mistym has joined #archiveteam [07:11] *** mistym has quit IRC (Read error: Operation timed out) [07:28] *** primus104 has quit IRC (Leaving.) [07:31] *** atomotic has joined #archiveteam [07:40] So uhh [07:40] How exactly does one go about extracting a .warc? [07:43] there's a bunch of tools to read them; if you want to extract response data, https://pypi.python.org/pypi/Warcat works [07:44] Hmm, I've just tried that actually [07:44] It doesn't extract anything [07:44] this warc file is 4kb [07:44] It was generated by wpull, and I am certain it is not meant to be 4kb [07:47] what's the WARC [07:47] What do you mean what's the WARC? [07:47] I need a URL to the WARC you are talking about [07:47] wpull generates many WARCs for a given grab [07:47] there's the content WARCs, and there can be one to many of those [07:47] depending on your split settings [07:47] then there's the meta-WARC which contains information about the grab [07:47] There is only one warc here [07:47] And I just used the example [07:48] Is there a way to make wpull output in a sane format [07:48] yes, WARC [07:48] ie: a directory tree [07:48] directory trees don't map well to request/response [07:48] I need the WARC to diagnose this problem, anything else is guessing [07:48] You can have the warc [07:48] one moment [07:49] one thing that is really pedantic but something I really want to convince people of is that WARC isn't recording files [07:49] it's recording HTTP requests and responses [07:49] I think I'm going to lose that but meh [07:49] It'd be nice if there was a way to record files [07:50] actually, hell, it's recording requests and responses for various protocols but yeah [07:50] https://appledash.org/warc.warc.gz Enjoy your WARC [07:51] yeah, that WARC's empty, so nothing was fetched [07:51] the only record in it is the wpull log [07:51] I think that much was fetched [07:51] Considering wpull talked about how it was fetching files for about 10 minutes [07:51] https://gist.github.com/anonymous/fb3ed4b5efc7058632fb [07:51] it's empty [07:52] Well... Why is it empty then? [07:52] ... [07:52] I just segfaulted wpull [07:52] What [07:52] Isn't it Python? [07:53] what's your command line [07:53] I ran it again and it started going again, do you still want my command line [07:54] ? [07:54] yes [07:54] segfaults can occur in any running process regardless of language [07:54] one of archivebot's larger nodes gets python3 segfaults a lot [07:54] *** phillipsj has joined #archiveteam [07:55] the Wpull-Arguments line provides the same information as the command line, but I find the command line is usually easier to read [07:55] Well yeah, but I'm fairly sure that a segfault triggered by a Py script (at least one that's not doing anything fishy with Python's internals) is a Python bug [07:56] https://gist.github.com/AppleDash/65c55986e224c4560bb3 [07:56] segfaults could be anything [07:57] I just ran it again after removing all files - the new warc is 7M which seems a bit more reasonable [07:57] ok, yeah [07:57] so that command will log data to irc.org.warc.gz [07:58] Define "log data" [07:58] log WARC records [07:59] you can peek at the WARC in progress with zcat/zless [07:59] http://ascii.textfiles.com/archives/4604 [08:00] The new file seems to contain the expected datas [08:00] data* [08:00] I wonder why the old one did not [08:01] wpull by default overwrites WARCs specified with --warc-file unless you specify --warc-append [08:01] so if you started another job writing to the new file and quickly terminated it that would result in a zero length file [08:03] appledash: is the command line you sent me the same one as the command line that yielded the WARC with the zero-fetch log? [08:04] Yes [08:04] yipdw, I see the problem them [08:04] I thought it would resume [08:04] --warc-append will do that [08:04] I stopped it to double the concurrency level [08:07] it may be useful to have wpull warn on overwrite [08:07] you'll probably want to talk to chfoo about that [08:07] well, warn or refuse to start [08:08] that would be useful [08:26] SketchCow: thank your for emulatity [08:26] *emularity [08:27] it fixes the problem with running java script emulator on a mesh network [08:27] cause if can run locally there is a way to the html/javascript on local wifi [08:54] *** godane has quit IRC (Ping timeout: 265 seconds) [09:14] *** godane has joined #archiveteam [09:33] *** primus104 has joined #archiveteam [10:13] *** VADemon has joined #archiveteam [10:20] *** rolfb has joined #archiveteam [10:43] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [10:46] *** schbirid has joined #archiveteam [11:19] *** beardicus has joined #archiveteam [11:33] *** Morbus has quit IRC (Quit: http://www.disobey.com/) [11:40] *** atomotic has joined #archiveteam [11:43] *** Morbus has joined #archiveteam [11:44] *** primus104 has quit IRC (Leaving.) [11:45] *** dugo has quit IRC (Read error: Operation timed out) [11:54] *** dugo has joined #archiveteam [12:10] *** rolfb has quit IRC (Leaving...) [12:14] *** BlueMaxim has quit IRC (Quit: Leaving) [12:20] *** rolfb has joined #archiveteam [12:48] *** sankin has joined #archiveteam [13:02] *** VADemon has quit IRC (Read error: Connection reset by peer) [13:06] *** Ymgve has joined #archiveteam [13:14] *** sankin has quit IRC (Leaving.) [13:22] *** julian2 has joined #archiveteam [13:22] *** rolfb has quit IRC (Read error: Connection reset by peer) [13:26] *** sankin has joined #archiveteam [13:52] *** aNthraXx_ has joined #archiveteam [13:52] *** aNthraXx has quit IRC (Read error: Operation timed out) [13:56] *** Start has quit IRC (Disconnected.) [14:00] *** primus104 has joined #archiveteam [14:09] *** Rickster has quit IRC (Ping timeout: 260 seconds) [14:11] *** primus104 has quit IRC (Leaving.) [14:12] *** Rickster has joined #archiveteam [14:20] *** rolfb has joined #archiveteam [14:33] *** Start has joined #archiveteam [14:34] *** mistym has joined #archiveteam [14:53] *** rolfb has quit IRC (Leaving...) [14:57] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [14:58] *** Start has quit IRC (Disconnected.) [15:02] *** Start has joined #archiveteam [15:51] *** Start has quit IRC (Disconnected.) [15:58] *** Start has joined #archiveteam [16:02] *** mistym has quit IRC (Remote host closed the connection) [16:28] *** aaaaaaaaa has joined #archiveteam [16:34] *** mistym has joined #archiveteam [16:37] *** mutoso has quit IRC (Read error: Connection reset by peer) [16:38] *** Start has quit IRC (Disconnected.) [16:38] *** mistym has quit IRC (Client Quit) [16:50] *** Start has joined #archiveteam [16:56] *** philpem has joined #archiveteam [16:59] *** scyther has joined #archiveteam [17:18] *** dashcloud has quit IRC (Read error: Operation timed out) [17:30] *** dashcloud has joined #archiveteam [17:40] *** SimpBrain has joined #archiveteam [17:42] *** Start has quit IRC (Disconnected.) [17:46] *** K4k has joined #archiveteam [17:49] *** primus104 has joined #archiveteam [17:57] *** MorbusIff has joined #archiveteam [17:57] *** Morbus has quit IRC (Ping timeout: 258 seconds) [18:27] *** mistym has joined #archiveteam [18:28] *** mistym has quit IRC (Client Quit) [19:29] *** Start has joined #archiveteam [19:49] *** SN4T14_ has joined #archiveteam [19:55] *** SN4T14 has quit IRC (Ping timeout: 369 seconds) [20:11] *** NovaKing_ has quit IRC (Read error: Operation timed out) [20:11] *** schbirid has quit IRC (Leaving) [20:11] *** SimpBrain has quit IRC (Quit: Leaving) [20:19] *** Start has quit IRC (Disconnected.) [20:38] *** rolfb has joined #archiveteam [20:40] *** NovaKing_ has joined #archiveteam [20:41] *** aaaaaaaaa has quit IRC (Leaving) [20:45] *** mistym has joined #archiveteam [20:52] *** aaaaaaaaa has joined #archiveteam [20:53] *** sankin has quit IRC (Leaving.) [21:06] *** scyther has quit IRC (Read error: Connection reset by peer) [21:08] *** dashcloud has quit IRC (Read error: Operation timed out) [21:17] *** dashcloud has joined #archiveteam [21:19] *** dashcloud has quit IRC (Read error: Operation timed out) [21:24] *** nertzy has joined #archiveteam [21:25] *** dashcloud has joined #archiveteam [21:26] *** BlueMaxim has joined #archiveteam [21:32] *** zenguy_pc has quit IRC (Ping timeout: 258 seconds) [21:36] *** rolfb has quit IRC (Leaving...) [21:37] *** signius has quit IRC (Read error: Operation timed out) [21:39] *** zenguy_pc has joined #archiveteam [21:52] *** signius has joined #archiveteam [21:52] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [22:08] *** garyrh has quit IRC (Ping timeout: 506 seconds) [22:17] *** Start has joined #archiveteam [22:18] *** julian2 has quit IRC (Quit: Leaving) [22:30] *** ersi has quit IRC (Ping timeout: 512 seconds) [22:31] *** ohhdemgir has joined #archiveteam [22:40] *** philpem has quit IRC (Ping timeout: 260 seconds) [22:49] this might be interesting if it's anything like what's being described: http://www.forbes.com/sites/thomasbrewster/2015/04/10/darpa-memex-search-going-open-source-check-it-out/ (open source release planned on Friday) [23:01] *** Start_ has joined #archiveteam [23:01] *** Start has quit IRC (Read error: Connection reset by peer) [23:48] *** Start has joined #archiveteam [23:48] *** Start_ has quit IRC (Read error: Connection reset by peer) [23:50] *** dashcloud has quit IRC (Read error: Operation timed out) [23:54] *** Stiletto has quit IRC () [23:59] *** dashcloud has joined #archiveteam