[00:03] *** Stiletto has joined #archiveteam-ot [01:31] *** noirscape has quit IRC (Quit: ZNC 1.7.1 - https://znc.in) [01:31] *** argus_ has quit IRC (Read error: Connection reset by peer) [01:33] *** argus has joined #archiveteam-ot [01:33] *** noirscape has joined #archiveteam-ot [02:27] *** picklefac has joined #archiveteam-ot [02:52] *** kiska1 has quit IRC (Read error: Operation timed out) [02:52] *** kiska1 has joined #archiveteam-ot [04:07] *** Hani111 has joined #archiveteam-ot [04:07] *** Hani has quit IRC (Read error: Connection reset by peer) [04:07] *** Hani111 is now known as Hani [04:09] *** logchfoo2 has quit IRC (Ping timeout: 252 seconds) [04:10] *** logchfoo3 starts logging #archiveteam-ot at Tue Feb 19 04:10:10 2019 [04:10] *** logchfoo3 has joined #archiveteam-ot [04:14] *** godane has quit IRC (Leaving.) [04:15] *** chirlu has quit IRC (Read error: Operation timed out) [04:28] *** chirlu has joined #archiveteam-ot [04:30] *** w00dsman has joined #archiveteam-ot [04:31] *** odemg has quit IRC (Ping timeout: 615 seconds) [04:36] *** w00dsman has quit IRC (Leaving) [04:38] *** odemg has joined #archiveteam-ot [04:45] *** Albardin has quit IRC (Read error: Operation timed out) [04:45] *** Albardin has joined #archiveteam-ot [04:45] *** kiskabak has quit IRC (Quit: Ping timeout (120 seconds)) [04:45] *** kiskabak has joined #archiveteam-ot [04:56] *** m007a83_ has joined #archiveteam-ot [04:59] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [06:50] *** Stiletto has quit IRC () [07:10] *** Stiletto has joined #archiveteam-ot [07:43] *** thewisefl has joined #archiveteam-ot [07:47] *** wise_flow has quit IRC (Read error: Operation timed out) [08:04] *** picklefac has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…) [08:16] *** wp494 has quit IRC (Ping timeout: 615 seconds) [08:17] *** wp494 has joined #archiveteam-ot [08:37] *** schbirid has joined #archiveteam-ot [10:20] *** BlueMax has quit IRC (Quit: Leaving) [10:31] *** picklefac has joined #archiveteam-ot [10:48] *** picklefac has quit IRC (Read error: Connection reset by peer) [10:49] *** picklefac has joined #archiveteam-ot [11:14] *** wise_flow has joined #archiveteam-ot [11:17] *** thewisefl has quit IRC (Read error: Operation timed out) [12:07] *** Albardin has quit IRC (Read error: Connection reset by peer) [12:08] *** kiskabak has quit IRC (Ping timeout: 265 seconds) [14:06] Can someone help me scrape this site? https://www.oldtimeradiodownloads.com/all-shows?display=2650 [14:06] ie, https://www.oldtimeradiodownloads.com/thriller/zero-hour/zero-hour-74-02-11-041-someones-death-chapter-1 [14:08] how does 'Content-Disposition: filename="zero-hour-closed-circuit-press-conference-1973-11-01.mp3"' work? [14:08] the download path is always 'https://www.oldtimeradiodownloads.com/player/audio.php' [14:11] *** icedice has joined #archiveteam-ot [14:24] Raccoon: The Content-Disposition is the server's recommendation for a filename. I think wget ignores it by default (due to security concerns), but there's an option to enable using it. Don't remember what it's called though. [14:24] ah right. [14:25] actually, seems to be that audio.php might be referencing Referer: https://www.oldtimeradiodownloads.com/thriller/zero-hour/zero-hour-closed-circuit-press-conference-1973-11-01 [14:25] and I have no idea how to pull a list of page names off this site. wget spider doesn't do it. [14:26] Yeah, audio.php seems to work like that. What about https://www.oldtimeradiodownloads.com/download/get_file/13872 ? [14:26] how do you come by that link [14:27] That link appears in the HTML but isn't displayed by default. Site seems sketchy, so I won't enable JS for it to see when it appears. It looks like there's some "your download will start in X seconds" timer thing though. [14:27] that downloaded right away. [14:28] the site, as far as I can tell, doesn't let you download files. you can either use the embedded player or pay to download [14:28] *** godane has joined #archiveteam-ot [14:28] well dang man! https://www.oldtimeradiodownloads.com/download/get_file/1 https://www.oldtimeradiodownloads.com/download/get_file/2 etc [14:28] :-) [14:28] thanks! [14:29] i'll just iterate a file list for wget, and turn on content disposition [14:29] Won't get you the metadata though unless it's embedded in the tags. [14:30] So you might still want to scrape the site for that. [14:30] And be prepared for IP bans. [14:30] you mean program descriptions? [14:31] Yeah [14:31] Title, air date, etc. [14:31] The filenames will most likely not be consistent. [14:31] yeah, i wouldn't know how to do that nicely [14:31] will have to see. so far they seem to be named sanely [14:33] the all-shows page episode totals to 77212 [14:42] *** chimyatta has joined #archiveteam-ot [14:45] thanks again man, so far so good. see how far this gets. [14:55] ah shit. got to 50 and now it 500's on me [14:55] even gave it a --wait 3 [15:00] oh. looks like gaps in the number sequence, and raises a 500 error in those gaps [15:08] *** Mateon1 has quit IRC (Ping timeout: 615 seconds) [15:09] *** Mateon1 has joined #archiveteam-ot [15:12] *** godane has quit IRC (Leaving.) [16:36] *** yano has quit IRC (Quit: WeeChat, The Better IRC Client, https://weechat.org/) [16:41] *** yano has joined #archiveteam-ot [17:11] *** wp494 has quit IRC (Ping timeout: 364 seconds) [17:12] *** wp494 has joined #archiveteam-ot [17:23] *** Fusl has quit IRC (Read error: Operation timed out) [17:27] *** Fusl has joined #archiveteam-ot [17:52] Fusl: i kind of wish this project was on freenode; as that is my camping grounds [17:52] but meh, i'm sure the people running this have a reason [17:53] heh, that gets thrown around a lot [17:53] biggest reason probably is that they don't want to migrate hundreds of people over to another network lol [17:54] It's pretty much just inertia at this point [17:54] The reason is "we've always been here", basically. Moving channels is annoying enough, moving networks is even worse. [17:54] efnet being a bit.. lax on 'ownership' and control of channels is a blessing and a curse [18:05] *** Stiletto has quit IRC (Ping timeout: 252 seconds) [18:07] *** Stiletto has joined #archiveteam-ot [18:11] *** Despatche has joined #archiveteam-ot [18:26] *** picklefac has quit IRC (Quit: My MacBook has gone to sleep. ZZZzzz…) [18:35] *** step has quit IRC (Read error: Operation timed out) [18:40] *** step has joined #archiveteam-ot [18:50] *** picklefac has joined #archiveteam-ot [19:37] *** SimpBrain has joined #archiveteam-ot [20:33] *** wise_flow has quit IRC (Remote host closed the connection) [20:36] *** wiseflowe has joined #archiveteam-ot [20:37] *** wiseflowe has quit IRC (Remote host closed the connection) [20:37] *** wiseflowe has joined #archiveteam-ot [20:39] *** wiseflowe has quit IRC (Remote host closed the connection) [20:39] *** wiseflowe has joined #archiveteam-ot [20:40] *** wise_flow has joined #archiveteam-ot [20:42] *** wise_flow has quit IRC (Remote host closed the connection) [20:44] *** wiseflowe has quit IRC (Ping timeout: 252 seconds) [20:45] *** wise_flow has joined #archiveteam-ot [20:46] *** thewisefl has joined #archiveteam-ot [20:47] *** thewisefl has quit IRC (Remote host closed the connection) [20:47] *** thewisefl has joined #archiveteam-ot [20:48] *** thewisefl has quit IRC (Remote host closed the connection) [20:49] *** thewisefl has joined #archiveteam-ot [20:49] *** wise_flow has quit IRC (Ping timeout: 252 seconds) [20:50] *** icedice has quit IRC (Quit: Leaving) [20:50] *** thewisefl has quit IRC (Remote host closed the connection) [20:50] *** thewisefl has joined #archiveteam-ot [20:51] *** thewisefl has quit IRC (Remote host closed the connection) [20:53] *** thewisefl has joined #archiveteam-ot [20:54] *** thewisefl has quit IRC (Remote host closed the connection) [20:54] *** thewisefl has joined #archiveteam-ot [20:55] *** thewisefl has quit IRC (Remote host closed the connection) [20:55] *** thewisefl has joined #archiveteam-ot [20:57] *** thewisefl has quit IRC (Remote host closed the connection) [20:57] *** thewisefl has joined #archiveteam-ot [21:00] *** thewisefl has quit IRC (Remote host closed the connection) [21:00] *** thewisefl has joined #archiveteam-ot [21:13] *** icedice has joined #archiveteam-ot [21:40] *** BlueMax has joined #archiveteam-ot [23:25] *** m007a83_ has quit IRC (Ping timeout: 252 seconds) [23:28] *** m007a83 has joined #archiveteam-ot [23:46] I like EFNet and I'm staying. [23:46] Don't do it! [23:47] oh, yano is spamming his network again :) [23:47] Raccoon: it's not *my* network [23:47] staffers gonna staff [23:47] i'm not a staffer [23:48] you were one [23:48] 5 years ago [23:48] for less than 2-years [23:48] Raccoon: sounds like you are stuck in the past :p [23:48] spammers gonna spam [23:52] also pretty sure pirating copyright content is a violation of freenode's such n such [23:54] :::: COPYRIGHT :::: [23:54] we don't talk about that word