[00:10] *** tomwsmf has joined #archiveteam-bs [00:14] *** JesseW has joined #archiveteam-bs [00:31] *** REiN^ has quit IRC (Read error: Operation timed out) [00:33] *** ravetcofx has joined #archiveteam-bs [01:22] *** schbirid has quit IRC (Ping timeout: 258 seconds) [01:30] *** wp494 has quit IRC (Ping timeout: 250 seconds) [01:36] *** schbirid has joined #archiveteam-bs [01:38] *** Ravenloft has joined #archiveteam-bs [01:51] *** twrist has joined #archiveteam-bs [01:52] *** GLaDOS has quit IRC (Ping timeout: 260 seconds) [01:53] *** twrist has quit IRC (Read error: Operation timed out) [01:54] *** GLaDOS has joined #archiveteam-bs [02:04] *** wp494 has joined #archiveteam-bs [02:24] i'm uploading more of 'The Times News (Idaho Newspaper)' pdfs [02:24] 1998 to 2000 for now [02:27] btw i'm at 778k items [02:49] *** GLaDOS has quit IRC (Quit: Oh crap, I died.) [02:49] *** GLaDOS has joined #archiveteam-bs [02:57] *** JesseW has quit IRC (Ping timeout: 370 seconds) [03:02] !a http://savehorizons.tripod.com/ --no-offsite-links [03:15] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [03:15] *** Ravenloft has joined #archiveteam-bs [03:16] *** tomwsmf has quit IRC (Read error: Operation timed out) [03:30] *** Discant has joined #archiveteam-bs [03:30] *** chazchaz_ has joined #archiveteam-bs [03:31] *** chazchaz_ has quit IRC (Remote host closed the connection) [03:35] *** chazchaz_ has joined #archiveteam-bs [03:36] *** chazchaz_ has quit IRC (Remote host closed the connection) [03:39] *** xfce has quit IRC (Quit: .) [03:40] *** chazchaz_ has joined #archiveteam-bs [03:41] *** JesseW has joined #archiveteam-bs [03:42] *** Discant has quit IRC (Read error: Operation timed out) [03:42] *** chazchaz_ has quit IRC (Read error: Connection reset by peer) [03:43] *** chazchaz_ has joined #archiveteam-bs [03:43] *** chazchaz_ has quit IRC (Remote host closed the connection) [03:52] i feel like some one should give Boardwatch Magazine Collection to bombjack [03:52] there are only like 5 issues on there site [03:53] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [03:53] *** chazchaz_ has joined #archiveteam-bs [03:54] *** chazchaz_ has quit IRC (Remote host closed the connection) [03:55] *** chazchaz_ has joined #archiveteam-bs [03:56] *** Panasoul_ has joined #archiveteam-bs [03:57] *** chazchaz_ has quit IRC (Remote host closed the connection) [03:58] *** chazchaz_ has joined #archiveteam-bs [04:04] *** JesseW has quit IRC (Ping timeout: 370 seconds) [04:15] *** Discant has joined #archiveteam-bs [04:16] *** nightpool has joined #archiveteam-bs [04:21] *** marvinw is now known as ivan [04:30] *** Discant has quit IRC (Read error: Connection reset by peer) [04:30] *** JesseW has joined #archiveteam-bs [04:48] *** Panasoul_ has quit IRC (Ping timeout: 246 seconds) [04:48] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [04:49] *** DoomTay has quit IRC (Quit: Page closed) [04:50] *** tomwsmf has joined #archiveteam-bs [04:53] *** DoomTay has joined #archiveteam-bs [04:54] *** GLaDOS has quit IRC (Read error: Connection reset by peer) [04:54] *** GLaDOS has joined #archiveteam-bs [04:54] *** Sk1d has joined #archiveteam-bs [05:18] *** nightpool has quit IRC (Read error: Operation timed out) [05:37] *** metalcamp has joined #archiveteam-bs [05:59] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [06:02] *** metalcamp has joined #archiveteam-bs [06:07] *** Aranje has joined #archiveteam-bs [06:07] *** yipdw has quit IRC (Remote host closed the connection) [06:12] *** DoomTay has quit IRC (Quit: Page closed) [06:14] *** REiN^ has joined #archiveteam-bs [06:17] *** ndiddy has quit IRC (Read error: Operation timed out) [06:24] *** JesseW has quit IRC (Ping timeout: 370 seconds) [06:42] *** tomwsmf has quit IRC (Read error: Operation timed out) [06:46] *** Aranje has quit IRC (Quit: Three sheets to the wind) [06:53] *** yipdw has joined #archiveteam-bs [06:55] *** ravetcofx has quit IRC (Remote host closed the connection) [06:58] *** FalconK has quit IRC (Ping timeout: 260 seconds) [06:59] *** GLaDOS has quit IRC (Quit: Oh crap, I died.) [06:59] *** GLaDOS has joined #archiveteam-bs [07:02] *** ravetcofx has joined #archiveteam-bs [07:07] *** FalconK has joined #archiveteam-bs [08:00] *** GLaDOS has quit IRC (Read error: Operation timed out) [08:00] *** GLaDOS has joined #archiveteam-bs [08:03] *** DiscantX has joined #archiveteam-bs [08:11] *** GLaDOS has quit IRC (Read error: Connection reset by peer) [08:11] *** twrist has joined #archiveteam-bs [08:12] *** twrist is now known as GLaDOS [08:14] so i'm grabbing BBC Radio 4 Today [08:14] at least whats there [08:16] github's unlimited repositories working great for some: https://github.com/kanripo :P [08:23] *** GLaDOS has quit IRC (Read error: Connection reset by peer) [08:25] *** GLaDOS has joined #archiveteam-bs [08:31] that's so bizarre, why one file per repo [08:33] oh one text per repository [08:33] that makes a bit more sense [08:41] so BBC Radio 4 The World Tonight has some good archives [08:41] going back to 2009-04-14 [09:20] *** GLaDOS has quit IRC (Oh crap, I died.) [09:20] *** GLaDOS has joined #archiveteam-bs [09:58] *** GLaDOS has quit IRC (Quit: Oh crap, I died.) [10:01] *** GLaDOS has joined #archiveteam-bs [10:06] *** GLaDOS has quit IRC (Quit: Oh crap, I died.) [10:06] *** GLaDOS has joined #archiveteam-bs [10:12] *** GLaDOS has quit IRC (Oh crap, I died.) [10:12] *** GLaDOS has joined #archiveteam-bs [11:06] lol http://www.pcgamer.com/company-claims-no-mans-sky-uses-its-patented-equation-without-permission/ [11:11] https://akp.thecthulhu.com/statement.txt [12:12] *** kristian_ has joined #archiveteam-bs [12:36] are you writing a script set for portalgraphics arkiver ? [12:36] yeah, almost done [12:37] Ah cool [13:27] question if anybody knows pls answe: does wpull in grab-site automatically splits warcs? [13:28] luckcolor: I'm not sure if it will do that without any set settings [13:28] But yes, it can do that [13:31] for the record "--warc-max-size=BYTES: Try to limit each WARC file to around BYTES bytes before rolling over to a new WARC file (default: 5368709120, which is 5GiB). Note that the resulting WARC files may be drastically larger if there are very large responses." [13:56] *** DoomTay has joined #archiveteam-bs [13:59] *** kristian_ has quit IRC (Leaving) [14:01] *** BlueMaxim has quit IRC (Quit: Leaving) [14:09] *** schbirid has quit IRC (Quit: Leaving) [14:19] So....arkiver, any progress on that warrior? [14:20] For which project DoomTay ? [14:20] portalgraphics [14:20] Should be finished soon I think [14:20] The "death date" is less than a week away [14:20] He's working on it now [14:20] Cool [14:21] I've juggled some scripts / stuff [14:21] So i can get a bunch of decent hosts pulling as soon as they're released [14:21] And it turns out that ArchiveBot made a bit of a mess with the relevant Jjb I fed it, so I'm going back and resaving a handful of things' [14:21] *** nightpool has joined #archiveteam-bs [15:17] *** JesseW has joined #archiveteam-bs [15:36] *** JesseW has quit IRC (Ping timeout: 370 seconds) [16:56] *** ndiddy has joined #archiveteam-bs [17:13] *** tomwsmf has joined #archiveteam-bs [17:25] *** schbirid has joined #archiveteam-bs [17:28] *** DoomTay has quit IRC (Quit: Page closed) [17:36] *** ravetcofx has quit IRC (Quit: Leaving) [17:44] *** ravetcofx has joined #archiveteam-bs [17:51] *** tomwsmf has quit IRC (Read error: Operation timed out) [18:10] *** DoomTay has joined #archiveteam-bs [18:16] *** kristian_ has joined #archiveteam-bs [18:27] *** GLaDOS has quit IRC (Quit: Oh crap, I died.) [18:27] *** GLaDOS has joined #archiveteam-bs [18:33] *** GLaDOS has quit IRC (Oh crap, I died.) [18:33] *** GLaDOS has joined #archiveteam-bs [18:42] *** GLaDOS has quit IRC (Ping timeout: 260 seconds) [18:56] *** kristian_ has quit IRC (Leaving) [19:40] DoomTay: can you please have a look at this log http://paste.nerds.io/raw/epirojifay [19:40] If it looks good to you, we'll start the grab [19:43] Can I see how you're handling numbers below 1000? [19:44] Everything seems okay, but it looks like attaching the lang= http://www.portalgraphics.net/pg/illust/?image_id=90331 does no good. [19:45] I'm going to keep the lang= in for now [19:45] if it turns out we can't grab everything in time I'll remove those URLs. [19:45] ID 139: http://paste.nerds.io/raw/cuqelebajo [19:46] Yup, looks good [19:46] And also a last-minute heads up, the upper limit is now 90346 and that may change tomorrow [19:48] running a private crawl on (http://www.intercambiosvirtuales.org) - a Spanish torrent/piracy site [19:50] Do you have a thing to catch occurances of "回線が込み合っております。時間をおいて再度アクセスしてください", which basically means "The servers are busy. Please wait"? [19:57] DoomTay: no, I never got that message [19:57] Hmm [19:57] do you have a status code for that? [19:57] It'll give you a 200 [20:01] Will the whole page be ??????????????????????????????? [20:01] or is that string somewhere on the page? [20:01] also, view-source:http://www.portalgraphics.net/pg/illust/?image_id=1 [20:01] It has the japanese characters and some more info after that [20:01] http://www.portalgraphics.net/pg/illust/?image_id=1 [20:02] does the servers are busy message also have some more explanation? [20:07] *** DoomTay has quit IRC (Ping timeout: 268 seconds) [20:07] *** kvieta has quit IRC (Read error: Operation timed out) [20:07] *** kvieta has joined #archiveteam-bs [20:07] *** zerkalo has quit IRC (Read error: Operation timed out) [20:08] *** hawc145 has joined #archiveteam-bs [20:08] *** SilSte has quit IRC (Read error: Operation timed out) [20:08] *** HCross has quit IRC (Read error: Operation timed out) [20:09] *** xmc has quit IRC (Read error: Operation timed out) [20:09] *** dashcloud has quit IRC (Read error: Operation timed out) [20:09] *** Fletcher has quit IRC (Read error: Operation timed out) [20:09] *** yeoldetoa has quit IRC (Read error: Operation timed out) [20:11] *** dxrt- has quit IRC (Read error: Operation timed out) [20:11] *** DoomTay has joined #archiveteam-bs [20:12] If I remember correctly, the only other thing besides that message was a link to www.portalgraphics.net [20:12] *** Start has quit IRC (Read error: Operation timed out) [20:12] *** brayden has quit IRC (Read error: Operation timed out) [20:12] As for that other thing you just linked me, that basically means the image was actually deleted [20:13] So yeah, in other news, my house kinda lost power, hence my sudden disappearance [20:13] *** dashcloud has joined #archiveteam-bs [20:14] *** yeoldetoa has joined #archiveteam-bs [20:14] *** Fusl has quit IRC (Read error: Operation timed out) [20:16] *** Fusl has joined #archiveteam-bs [20:18] *** JW_work has quit IRC (Ping timeout: 633 seconds) [20:19] DoomTay: you don't have a copy of the full response you got stored anywhere right? [20:19] I'm digging for such a page right now... [20:19] I know Wayback Machine happens to have such a thing [20:19] *** zerkalo has joined #archiveteam-bs [20:20] Oh, here's one: http://web.archive.org/web/20160718113152/http://portalgraphics.net/pg/illust/download.php?image_id=57054 [20:21] Thing is unicode is a little problematic in lua [20:22] Oh [20:22] (should move to python) [20:22] I'll check the received content for


[20:22] if it has that, grab will be aborted [20:22] Cool [20:22] *** goekesmi_ has quit IRC (Ping timeout: 633 seconds) [20:22] normal pages don't seem to have that string [20:26] *** Ravenloft has joined #archiveteam-bs [20:28] *** kristian_ has joined #archiveteam-bs [20:29] *** goekesmi has joined #archiveteam-bs [20:29] *** DoomTay has quit IRC (Ping timeout: 268 seconds) [20:31] *** Start has joined #archiveteam-bs [20:31] *** SilSte has joined #archiveteam-bs [20:32] *** DoomTay has joined #archiveteam-bs [20:35] But yeah, my house lost power and I'm on a laptop. I might disappear again in an hour or so. Just an FYI [20:39] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [20:42] *** VADemon has joined #archiveteam-bs [20:43] *** xmc has joined #archiveteam-bs [20:43] *** swebb sets mode: +o xmc [20:50] *** RichardG has quit IRC (Ping timeout: 260 seconds) [20:50] https://archive.org/details/bbc-radio-4-the-world-tonight-2009-04-14 [20:55] *** Fletcher has joined #archiveteam-bs [20:55] *** Fletcher_ sets mode: +o Fletcher [21:11] sort of good news everyone: https://web.archive.org/web/20150609001718im_/http://kat.cr/dailydump.txt.gz [21:12] kat was making dumps of there magnet urls last year [21:27] *** DoomTay has quit IRC (Ping timeout: 268 seconds) [21:28] *** DoomTay has joined #archiveteam-bs [21:42] *** RichardG has joined #archiveteam-bs [22:02] *** RichardG has quit IRC (Read error: Operation timed out) [22:04] *** RichardG has joined #archiveteam-bs [22:09] *** schbirid has quit IRC (Remote host closed the connection) [22:12] eighty bucks! fuck http://archiveteam.org/index.php?title=Restoring&curid=5695&diff=26063&oldid=25375 [22:27] *** GLaDOS has joined #archiveteam-bs [22:32] *** metalcamp has quit IRC (Ping timeout: 244 seconds) [22:39] *** JW_work has joined #archiveteam-bs [22:53] *** DiscantX has quit IRC (Read error: Operation timed out) [22:53] *** Sk1d has quit IRC (Ping timeout: 250 seconds) [22:53] *** w0rp has quit IRC (Ping timeout: 250 seconds) [22:53] *** espes__ has quit IRC (Ping timeout: 250 seconds) [22:53] *** Lord_Nigh has quit IRC (Ping timeout: 251 seconds) [22:53] *** dashcloud has quit IRC (Ping timeout: 250 seconds) [22:53] *** Fletcher_ has quit IRC (Ping timeout: 250 seconds) [22:53] *** w0rp has joined #archiveteam-bs [22:54] *** Sk1d has joined #archiveteam-bs [22:54] *** Lord_Nigh has joined #archiveteam-bs [22:54] *** dashcloud has joined #archiveteam-bs [22:55] *** Fletcher_ has joined #archiveteam-bs [22:56] https://techcrunch.com/2016/07/18/opera-renegotiates-its-1-2b-sale-down-to-600m-for-browser-privacy-apps-chinese-jv/ [22:57] Can we trust the company that they are selling to? Or is this going to be Adblock Plus all over again? [22:59] *** espes__ has joined #archiveteam-bs [23:06] *** kristian_ has quit IRC (Leaving) [23:08] *** BlueMaxim has joined #archiveteam-bs [23:14] *** dxrt- has joined #archiveteam-bs [23:14] *** dxrt sets mode: +o dxrt- [23:20] I didn't know ABP got sold to anyone [23:27] *** tomwsmf has joined #archiveteam-bs [23:38] *** nightpool has quit IRC (Ping timeout: 501 seconds) [23:43] Do you think http://www.centralfloridafuture.com/ would be too much for ArchiveBot? Might need saving given http://www.centralfloridafuture.com/story/news/2016/07/21/central-florida-future-cease-publication/87394312/ [23:58] According to google it has about "11,900 results"