[00:02] *** eLbot has joined #archiveteam-bs [00:04] *** zhongfu_ has quit IRC (Ping timeout: 260 seconds) [00:06] *** zhongfu has joined #archiveteam-bs [00:09] *** DoragonMo has joined #archiveteam-bs [00:09] *** DoragonMo has quit IRC (Remote host closed the connection) [00:17] *** BlueMax has joined #archiveteam-bs [00:34] *** dashcloud has joined #archiveteam-bs [00:43] *** ndiddy has joined #archiveteam-bs [00:44] *** RichardG_ has joined #archiveteam-bs [00:44] *** RichardG has quit IRC (Ping timeout: 260 seconds) [00:57] *** Petri152 has joined #archiveteam-bs [00:57] *** c4rc4s has joined #archiveteam-bs [00:57] *** wabu has joined #archiveteam-bs [00:57] *** zyphlar has joined #archiveteam-bs [01:00] *** JAA has joined #archiveteam-bs [01:00] *** swebb sets mode: +o JAA [01:00] *** bakJAA sets mode: +o JAA [01:00] *** jspiros has joined #archiveteam-bs [01:08] *** RichardG has joined #archiveteam-bs [01:08] *** RichardG_ has quit IRC (Read error: Connection reset by peer) [01:10] *** RichardG_ has joined #archiveteam-bs [01:10] *** RichardG has quit IRC (Read error: Connection reset by peer) [01:29] *** yuitimoth has quit IRC (Read error: Operation timed out) [01:33] *** godane has quit IRC (Read error: Operation timed out) [01:47] *** godane has joined #archiveteam-bs [01:47] *** svchfoo1 sets mode: +o godane [02:00] *** RichardG_ is now known as RichardG [02:48] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [02:49] *** ndiddy has quit IRC (Read error: Operation timed out) [03:06] *** odemg has quit IRC (Ping timeout: 260 seconds) [03:18] *** odemg has joined #archiveteam-bs [03:35] *** dashcloud has quit IRC (Read error: Operation timed out) [03:39] *** JAA has quit IRC (Read error: Operation timed out) [03:39] *** jspiros has quit IRC (Read error: Operation timed out) [03:40] *** zyphlar has quit IRC (Read error: Operation timed out) [03:41] *** Petri152 has quit IRC (Ping timeout: 246 seconds) [03:41] *** wabu has quit IRC (Ping timeout: 246 seconds) [03:42] *** c4rc4s has quit IRC (Read error: Operation timed out) [03:49] *** godane has quit IRC (Read error: Operation timed out) [04:09] *** godane has joined #archiveteam-bs [04:09] *** svchfoo3 sets mode: +o godane [04:23] *** vectr0n has quit IRC (ZNC - https://znc.in) [04:39] *** vectr0n has joined #archiveteam-bs [04:40] *** c4rc4s has joined #archiveteam-bs [04:40] *** zyphlar has joined #archiveteam-bs [04:41] *** Petri152 has joined #archiveteam-bs [04:42] *** wabu has joined #archiveteam-bs [04:44] *** JAA has joined #archiveteam-bs [04:44] *** swebb sets mode: +o JAA [04:44] *** bakJAA sets mode: +o JAA [04:44] *** jspiros has joined #archiveteam-bs [05:11] *** RichardG has quit IRC (Read error: Connection reset by peer) [05:11] *** RichardG has joined #archiveteam-bs [05:45] *** schbirid has joined #archiveteam-bs [06:06] *** Tenebrae has quit IRC (Ping timeout: 268 seconds) [06:06] *** Tenebrae has joined #archiveteam-bs [06:09] *** VADemon_m has quit IRC (Ping timeout: 268 seconds) [06:10] *** VADemon_m has joined #archiveteam-bs [06:10] *** VoynichCr has quit IRC (Ping timeout: 268 seconds) [06:12] *** gag45i8u8 has joined #archiveteam-bs [06:17] *** godane has quit IRC (Ping timeout: 268 seconds) [06:18] *** schbirid has quit IRC (Remote host closed the connection) [06:18] *** godane has joined #archiveteam-bs [06:19] *** svchfoo1 sets mode: +o godane [06:25] *** VoynichCr has joined #archiveteam-bs [06:25] I'm starting to upload Esquire Magazine [06:25] this is the full back catalog of that magazine [06:25] it will be in cbz format of course [06:26] SketchCow: also i found out about this: http://archive.dartmouthalumnimagazine.com/issues/1905 [06:26] i will start going after it cause its same way the Esquire magazine puts there issues online [06:31] Godane I may have some old magazine archives I found in open directories when I get home from work I will see if I still have them saved [06:34] cool [06:53] *** xmc has quit IRC (Quit: going to the desert - back after labor day) [07:21] *** yuitimoth has joined #archiveteam-bs [08:25] *** t2t2 has quit IRC (Read error: Operation timed out) [08:25] *** t2t2 has joined #archiveteam-bs [08:52] looks like macleans magazine can also be digitize from the same scripts: https://archive.macleans.ca/issues/1909 [08:54] *** wp494 has quit IRC (Read error: Operation timed out) [08:55] *** wp494 has joined #archiveteam-bs [09:11] i also found this one: https://archive.list.co.uk/ [09:50] *** caff_ has quit IRC (Read error: Connection reset by peer) [09:53] *** BlueMax has quit IRC (Quit: Leaving) [10:02] *** DragonMon has quit IRC (Quit: Leaving) [10:03] *** DragonMon has joined #archiveteam-bs [10:05] *** DragonMon has quit IRC (Client Quit) [10:05] *** DragonMon has joined #archiveteam-bs [10:07] *** DragonMon has quit IRC (Client Quit) [10:07] *** DragonMon has joined #archiveteam-bs [10:19] so i'm at 1.5 million items now [10:19] :-D [10:33] *** DragonMon has quit IRC (Quit: Leaving) [10:33] *** DragonMon has joined #archiveteam-bs [10:39] *** DragonMon has quit IRC (Quit: Leaving) [10:39] *** DragonMon has joined #archiveteam-bs [10:41] *** DragonMon has quit IRC (Client Quit) [10:42] *** DragonMon has joined #archiveteam-bs [10:47] Pushshift (the unofficial Reddit API which is way better than the official one) seems to be in financial trouble: https://old.reddit.com/r/pushshift/comments/988u25/pushshift_desperately_needs_your_help_with_funding/ [10:56] *** bitBaron has joined #archiveteam-bs [11:01] *** DragonMon has quit IRC (Quit: Leaving) [11:21] *** DragonMon has joined #archiveteam-bs [12:07] *** dashcloud has joined #archiveteam-bs [12:14] so i'm going to see about mirroring This Old House magazine : http://archive.thisoldhouse.com/ [12:18] *** dashcloud has quit IRC (Ping timeout: 492 seconds) [12:45] *** Mutter has joined #archiveteam-bs [12:48] *** Mutter has quit IRC (Mutter: www.mutterirc.com) [13:04] *** DragonMon has quit IRC (Quit: ZNC 1.7.1 - https://znc.in) [13:10] *** DragonMon has joined #archiveteam-bs [13:25] *** Mateon1 has quit IRC (Ping timeout: 252 seconds) [13:29] *** RichardG_ has joined #archiveteam-bs [13:29] *** RichardG has quit IRC (Read error: Connection reset by peer) [14:07] *** Mateon1 has joined #archiveteam-bs [14:22] *** bitBaron_ has joined #archiveteam-bs [14:24] *** sam-p has quit IRC (Read error: Connection reset by peer) [14:24] *** bitBaron has quit IRC (Read error: Operation timed out) [14:25] *** bitBaron has joined #archiveteam-bs [14:26] *** vitzli has joined #archiveteam-bs [14:26] *** bitBaro__ has joined #archiveteam-bs [14:26] *** bitBaron_ has quit IRC (Read error: Operation timed out) [14:29] *** sam-p has joined #archiveteam-bs [14:32] *** bitBaron has quit IRC (Read error: Operation timed out) [14:38] dxrt, thank you. It seems they are going doing on September, 1st, but right now they are being kind of ddos'ed - hug of death, they are quite big, ~10 000 topics, ~1 000 000 messages [14:39] aand forum sometimes gives 404 Not found from nginx [14:39] *** sam-p has quit IRC (Read error: Connection reset by peer) [14:46] Current numbers: 11.5k topics, 1.7M posts. [14:46] *** gag45i8u8 has quit IRC (Ping timeout: 260 seconds) [14:50] *** sam-p has joined #archiveteam-bs [14:57] *** schbirid has joined #archiveteam-bs [15:39] *** dashcloud has joined #archiveteam-bs [15:49] *** phuzion has quit IRC (Read error: Operation timed out) [15:50] *** SketchCow has quit IRC (Read error: Connection reset by peer) [15:50] *** SketchCow has joined #archiveteam-bs [15:50] *** swebb sets mode: +o SketchCow [15:51] *** phuzion has joined #archiveteam-bs [16:20] *** vitzli has quit IRC (Quit: Leaving) [16:24] *** superkuh has quit IRC (Remote host closed the connection) [16:25] *** superkuh has joined #archiveteam-bs [16:40] *** RichardG_ is now known as RichardG [17:07] *** ta9le has joined #archiveteam-bs [17:26] So yeah, I'm not sure if autodownloading things will be easy on Rom Hustler [17:27] not only all the download links appear to be generated for each hit, but they also don't work correctly on Opera - so maybe that means the crawlers as well [17:27] Yeah, won't be easy from looking at the source. [17:27] Crawlers will definitely not grab any of that due to the JS stuff. [17:27] the only good thing about all this is that there's no captcha [17:28] It sends an AJAX request to get the URL. [17:28] ps2 is well preserved by redump sets [17:28] Relevant code is near the bottom of http://romhustler.net/js/cache-1476400314-2281e1d8c88457259c7eac629cc2f46e.js . [17:29] and the others as well [17:29] i would say this is pointless [17:29] Yeah, ROMs aren't really the issue here, it's more about the communities around them. [17:30] ... which might continue to exist without ROM downloads (e.g. EmuParadise), but some might disappear entirely as well. [17:34] *** gafn00b17 has joined #archiveteam-bs [18:27] *** jut has quit IRC (Quit: WeeChat 1.4) [18:34] *** jut has joined #archiveteam-bs [18:40] *** ndiddy has joined #archiveteam-bs [18:42] *** VADemon_m has quit IRC (Ping timeout: 268 seconds) [18:49] *** VADemon_m has joined #archiveteam-bs [18:50] just look in devtools to see what requests it makes, and then emulate those [18:50] there's no need to evaluate or even read any JS in 99.99% of cases [18:51] Instagram being the one case where it is necessary. (You need to send the MD5 hash of a predefined value plus your scroll parameters in a header to scroll.) [18:52] But yeah, completely true. In this case though, the code is very readable. Could be worse. [19:08] *** caff_ has joined #archiveteam-bs [19:12] So posts on pastebin.mozilla.org use a numeric ID, which goes up to a bit over 9 million. By default, posts are deleted after a day, so most IDs will be invalid (not return any text). [19:13] URLs are https://pastebin.mozilla.org/9091995 for normal view and https://pastebin.mozilla.org/?dl=9091995 for download as a text file. (That's an example which was posted earlier today.) [19:21] JAA: [19:21] While we've been looking into this, we've also learned that nothing in Pastebin is older than 30 days - the "forever" option doesn't and apparently has never worked, and we've never noticed until now. [19:25] *** dashcloud has quit IRC (Read error: Operation timed out) [19:27] *** RichardG has quit IRC (Ping timeout: 360 seconds) [19:47] PurpleSym: huh? we talking about Pastebin.com? [19:48] or the Mozilla one? [19:52] *** ndiddy has quit IRC (Read error: Operation timed out) [19:54] *** Kenshin has quit IRC (Read error: Connection reset by peer) [19:55] *** Kenshin has joined #archiveteam-bs [20:04] *** chazchaz has quit IRC (Ping timeout: 260 seconds) [20:15] *** chazchaz has joined #archiveteam-bs [20:25] *** SmileyG has joined #archiveteam-bs [20:27] *** RichardG has joined #archiveteam-bs [20:27] *** Smiley has quit IRC (Ping timeout: 252 seconds) [20:27] *** dashcloud has joined #archiveteam-bs [20:30] *** schbirid has quit IRC (Remote host closed the connection) [20:32] *** godane has quit IRC (Ping timeout: 255 seconds) [20:38] *** RichardG has quit IRC (Read error: Connection reset by peer) [20:39] *** RichardG has joined #archiveteam-bs [20:46] *** RichardG_ has joined #archiveteam-bs [20:50] *** Polylith has quit IRC (Ping timeout: 268 seconds) [20:51] *** RichardG has quit IRC (Ping timeout: 506 seconds) [20:51] *** godane has joined #archiveteam-bs [20:51] *** svchfoo1 sets mode: +o godane [20:54] *** Polylith has joined #archiveteam-bs [20:57] PurpleSym: Wow, nice job, Mozilla... [20:58] (That could be said about a lot of things recently.) [21:00] It does explain why every single link I've found (except for the most recent ones) was broken. [21:02] And since it looks like the Pastebin was hardly used in the last few years (only 20k pastes since February https://support.mozilla.org/en-US/questions/1202904 -> https://pastebin.mozilla.org/9077235), there's really pretty much nothing that can be grabbed. [21:05] so my computer is acting weird [21:06] it did a random reboot with no waning [21:06] *warning [21:17] *** dashcloud has quit IRC (Read error: Operation timed out) [21:28] *** arbin_ has quit IRC (Remote host closed the connection) [21:29] *** arbin has joined #archiveteam-bs [21:31] *** faolingfa has joined #archiveteam-bs [21:40] *** gafn00b17 has quit IRC (Quit: Page closed) [21:49] *** dashcloud has joined #archiveteam-bs [22:03] *** zino has quit IRC (Read error: Operation timed out) [22:20] *** zino has joined #archiveteam-bs [22:46] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [22:47] *** ta9le has quit IRC (Quit: Connection closed for inactivity) [23:00] *** ndiddy has joined #archiveteam-bs [23:03] *** dan- has quit IRC (Ping timeout: 260 seconds) [23:10] https://www.mercatornet.com/harambee/view/closing-down-africas-internet/21641 [23:35] *** RichardG_ has quit IRC (Ping timeout: 268 seconds) [23:37] *** vitzli has joined #archiveteam-bs [23:46] *** RichardG has joined #archiveteam-bs [23:56] *** dan- has joined #archiveteam-bs