[00:00] *** adinbied has quit IRC (Quit: Left Channel.) [00:03] *** SN4T14_ has quit IRC (Quit: ZNC 1.6.3 - http://znc.in) [00:40] *** Sk1d has quit IRC (Read error: Operation timed out) [00:43] *** Sk1d has joined #archiveteam-bs [00:55] http://www.astronomy.com/news/2018/10/trove-of-newly-released-nasa-audio-puts-you-backstage-during-apollo-11 [00:55] Does IA now have this machine [01:47] *** Sk1d has quit IRC (Read error: Operation timed out) [01:50] *** Sk1d has joined #archiveteam-bs [02:04] *** Sk1d has quit IRC (Read error: Operation timed out) [02:06] *** Sk1d has joined #archiveteam-bs [02:39] *** Atom-- has joined #archiveteam-bs [02:42] *** Atom__ has quit IRC (Ping timeout: 252 seconds) [02:50] *** Sk1d has quit IRC (Read error: Operation timed out) [02:54] *** Sk1d has joined #archiveteam-bs [03:06] *** Sk1d has quit IRC (Read error: Operation timed out) [03:10] *** Sk1d has joined #archiveteam-bs [03:26] *** Sk1d has quit IRC (Read error: Operation timed out) [03:29] *** Sk1d has joined #archiveteam-bs [03:32] *** Stiletto has joined #archiveteam-bs [03:34] *** Stilett0 has quit IRC (Read error: Operation timed out) [04:27] *** Sk1d has quit IRC (Read error: Operation timed out) [04:31] *** Sk1d has joined #archiveteam-bs [04:43] *** Sk1d has quit IRC (Read error: Operation timed out) [04:47] *** Sk1d has joined #archiveteam-bs [05:01] *** Sk1d has quit IRC (Read error: Operation timed out) [05:04] *** Sk1d has joined #archiveteam-bs [05:06] *** jspiros has quit IRC (Read error: Operation timed out) [05:12] *** Stilett0 has joined #archiveteam-bs [05:13] *** jspiros has joined #archiveteam-bs [05:16] *** Stiletto has quit IRC (Read error: Operation timed out) [05:18] *** Sk1d has quit IRC (Read error: Operation timed out) [05:20] *** Sk1d has joined #archiveteam-bs [05:33] *** Sk1d has quit IRC (Read error: Operation timed out) [05:36] *** Sk1d has joined #archiveteam-bs [06:04] *** bsmith093 has quit IRC (Read error: Operation timed out) [06:09] *** bsmith093 has joined #archiveteam-bs [06:22] *** Sk1d has quit IRC (Read error: Operation timed out) [06:26] *** Sk1d has joined #archiveteam-bs [06:30] *** Atom__ has joined #archiveteam-bs [06:33] *** Atom-- has quit IRC (Ping timeout: 252 seconds) [07:31] *** Sk1d has quit IRC (Read error: Operation timed out) [07:35] *** Sk1d has joined #archiveteam-bs [08:10] *** schbirid has joined #archiveteam-bs [08:20] *** m007a83 has quit IRC (Read error: Operation timed out) [08:24] *** Sk1d has quit IRC (Read error: Operation timed out) [08:28] *** Sk1d has joined #archiveteam-bs [08:36] *** m007a83 has joined #archiveteam-bs [08:40] *** Sk1d has quit IRC (Read error: Operation timed out) [08:43] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [08:43] *** Sk1d has joined #archiveteam-bs [09:08] *** m007a83 has joined #archiveteam-bs [09:47] *** Sk1d has quit IRC (Read error: Operation timed out) [09:51] *** Sk1d has joined #archiveteam-bs [10:05] *** trhth has joined #archiveteam-bs [10:05] *** trhth has left [10:05] *** Sk1d has quit IRC (Read error: Operation timed out) [10:08] *** Sk1d has joined #archiveteam-bs [10:22] *** Sk1d has quit IRC (Read error: Operation timed out) [10:25] *** Sk1d has joined #archiveteam-bs [10:38] *** Sk1d has quit IRC (Read error: Operation timed out) [10:41] *** Sk1d has joined #archiveteam-bs [10:49] *** BlueMax has quit IRC (Quit: Leaving) [10:54] *** Sk1d has quit IRC (Read error: Operation timed out) [10:57] *** Sk1d has joined #archiveteam-bs [11:53] *** Sk1d has quit IRC (Read error: Operation timed out) [11:57] *** Sk1d has joined #archiveteam-bs [13:03] SketchCow: i'm now finally capturing the america's got talent finale tape [13:04] it was one of the tapes i pushed off cause its not a very old broadcast tape [13:08] *** Stiletto has joined #archiveteam-bs [13:09] *** Stilett0 has quit IRC (Read error: Operation timed out) [16:13] Oh lovely [16:56] *** Mateon1 has quit IRC (Read error: Operation timed out) [16:56] *** Mateon1 has joined #archiveteam-bs [17:10] Can someone queue forums.cracked.com on archive bot? [17:10] some of the sub forums close today [17:11] mundus20-: We already have a job for it, e9bwrlokmce28vq832rb5cgr. [17:11] awesome thanks [17:11] I'm wondering though whether I should send another few million requests their way to ensure that the entire thing is covered. [17:13] Only a few 100k actually. [17:13] Some of the sub forums get closed at 6PM CST [17:13] *EST [17:13] EDT? [17:15] DST ends on Sunday in the US, I think, so it should still be EDT. [17:15] In other words, 22:00 UTC, or a bit less than 5 hours. [17:15] yep [17:15] that [17:15] lol [17:37] *** lindalap has joined #archiveteam-bs [17:57] *** Sk1d has quit IRC (Read error: Operation timed out) [17:58] *** SmoothWil has joined #archiveteam-bs [17:58] Hello! [17:58] (Continuing from #archivebot regarding the Cracked forums) [18:00] SmoothWil: I don't really have time right now to set anything complicated up which targets the subforums specifically. I will simply bruteforce all ~194k thread IDs. [18:01] *** Sk1d has joined #archiveteam-bs [18:02] Gotcha. I thought it could count the subforums as domains, like site.com/viewforum=1 and so on. Thank you. [18:03] *** Stilett0 has joined #archiveteam-bs [18:03] A friend posted almost as much there as I did before dying suddenly, so it's a personal quest for me to make sure as much of him is saved as possible. [18:04] *** Stiletto has quit IRC (Read error: Operation timed out) [18:06] I asked the forum admin for more time and got no response, so I've been stressing about this for about 48 hours [18:06] Thank you. [18:08] SmoothWil: Do you know if the thread pagination is always 20 posts per page by default? [18:08] (I'm asking because I've seen forums where that setting varies per subforum.) [18:14] yes, it is [18:20] Looks like someone is trashing sessions on there. I wonder if it's ArchiveBot. [18:20] (Sessions expire extremely fast, within a few minutes.) [18:21] is that a bad thing? [18:23] It's a bit annoying because it means that session IDs are appearing everywhere, so links will be broken on playback in the Wayback Machine. [18:28] oh no! Is there anyway to sync them manually afterwards? or would running them through a forum profile help? (no idea what I'm talking about when it comes to archiving)? [18:28] the data is still just as saved, right? [18:40] Aaand I'm getting 503 "Server overloaded" already. [18:42] damn it [18:42] And as suspected, they're returning wrong status codes. Specifically, threads that require a login still return status 200 instead of 401. [18:44] uh-oh [18:44] does this mean it's hopeless? [18:45] It seems that 503 is their way of saying "you're banned, fuck off". [18:45] I.e. not an actual server overload. [18:46] Yeah, seems pretty hopeless unless that ban expires quickly. [18:50] I don't think it is? Are you using a proxy? The forum's been around so long that most of them are banned. [18:54] No proxy. The ban has expired now. 15 minutes probably. [18:55] so there's no way to tell archivebot to archive this (http://forums.cracked.com/viewforum.php?f=17) and everything "under" it separately? 'cause it looks as if right now archivebot is caught up in the image-heavy and less-endangered-for-now parts of the forums [18:56] ah, got word back: 503 error is "Load balance. Too many attempts likely another ddos" [18:57] With some magic (i.e. complex ignore patterns), it's possible to get close. But there's another problem there, which is that ArchiveBot has a bit of a queue right now, so new jobs won't start anytime soon. Certainly not before the 3-hour deadline. [18:58] damn. [19:16] 10 requests per second is already too fast apparently, and that's already too low a rate to grab everything in time. Welp [19:17] mundus20-, SmoothWil: Will those subforums that get "closed" today become inaccessible or just locked? [19:21] i do not know [19:23] *** _Verified has joined #archiveteam-bs [19:23] closed/deleted, I'm pretty sure [19:23] let me check [19:24] oooooh, intriguing! they are NOT [19:24] or... wait [19:26] *** VerifiedJ has quit IRC (Read error: Operation timed out) [19:27] okay, actual deletion is in Dec./Jan., but they are being hidden today/Nov. [19:29] *** lindalap has quit IRC (Quit: lindalap) [19:29] so yeah, invisible without an admin account [19:31] Write and ask for special access? [19:33] I did; the forum owner's a tool [19:33] like one of his mods died and he doesn't even care about saving even just those posts [19:50] Hmm, crap. [19:52] SmoothWil: Alright then. Do you have a full list of the forums that get closed tonight? [19:52] Yes! [19:53] I need the forum IDs, i.e. the "f" parameter on viewforum.php. [19:53] From most important to least important: [19:54] http://forums.cracked.com/viewforum.php?f=17 [19:54] http://forums.cracked.com/viewforum.php?f=16 [19:55] http://forums.cracked.com/viewforum.php?f=27 [19:55] http://forums.cracked.com/viewforum.php?f=29 [19:56] http://forums.cracked.com/viewforum.php?f=31 [19:56] http://forums.cracked.com/viewforum.php?f=15 [19:56] http://forums.cracked.com/viewforum.php?f=32 [19:57] http://forums.cracked.com/viewforum.php?f=28 [19:57] http://forums.cracked.com/viewforum.php?f=32 [19:58] *** _Verified is now known as VerifiedJ [19:58] and there's a couple more but they're really small [19:58] and unimportant [19:58] the first two are the most important (and largest) [20:10] *** Sk1d has quit IRC (Read error: Operation timed out) [20:12] *** wp494 has quit IRC (Read error: Operation timed out) [20:13] *** wp494 has joined #archiveteam-bs [20:14] *** Sk1d has joined #archiveteam-bs [20:18] does that help, JAA? [20:18] SmoothWil: Yeah, thanks. I started an ArchiveBot job for 16 and 17. [20:18] Not sure how much it'll grab, but it's better than nothing. [20:19] Thank you SO MUCH [20:19] I'll start another one for the others now. [20:19] You have no idea how less-guilty that makes me feel. [20:20] that I wasn't in-the-know enough to save it all. [20:20] SmoothWil: Hmm, you listed f=32 twice. [20:22] let me make sure one isn't supposed to be something else [20:24] Welp, the ArchiveBot job got banned as well, dammit. [20:24] 503? [20:24] Yeah [20:25] can you access the site over ipv6 at all? [20:25] Me? [20:25] jrwr: There's no AAAA record. [20:25] damn [20:29] oh shit, I still have HTTracker scanning one of the subforums at 1 connection per second. fuck ME [20:30] If I stopped that I assume that might help? [20:30] *** Sk1d has quit IRC (Read error: Operation timed out) [20:32] SmoothWil: Their servers can certainly handle one request per second. Also, as mentioned, the 503s aren't actually server-side issues most likely. [20:32] Basically, they ban IPs that exceed their rate limits and return a 503 page instead for those while the ban is active. [20:32] fuuuuuuuuck [20:33] *** Sk1d has joined #archiveteam-bs [21:18] *** Stiletto has joined #archiveteam-bs [21:20] *** Stilett0 has quit IRC (Ping timeout: 268 seconds) [21:22] *** Sk1d has quit IRC (Read error: Operation timed out) [21:26] *** Sk1d has joined #archiveteam-bs [21:27] *** SmoothWil has quit IRC (Quit: Page closed) [21:31] SketchCow: i'm uploading the Americas Got Talent finale episode [21:32] its incomplete but is woc [21:32] it was the only thing on the tape [21:38] *** BlueMax has joined #archiveteam-bs [22:11] *** Sk1d has quit IRC (Read error: Operation timed out) [22:16] *** Sk1d has joined #archiveteam-bs [22:22] SketchCow: i may have fixed the rewind problem with one of the tapes [22:33] nevermind [22:33] i can't get a good image of tape anyways [22:34] this is what i mean by no good image: https://imgur.com/a/TqHvU20 [23:34] *** VerifiedJ has quit IRC (Quit: Leaving) [23:35] *** VerifiedJ has joined #archiveteam-bs [23:38] *** Sk1d has quit IRC (Read error: Operation timed out) [23:43] *** Sk1d has joined #archiveteam-bs [23:59] *** Sk1d has quit IRC (Read error: Operation timed out)