[00:03] <Gallifrey> That's good to know. My home-made script, even if it worked perfectly, was only going to archive the top 1000 lists, plus a list of threads that I personally found interesting.
[00:03] *** BlueMax has joined #archiveteam-bs
[00:04] <Gallifrey> The latter are few enough in number that I can probably still do them with my script.
[00:06] <Gallifrey> I'll just have to archive them less frequently - I don't think Reddit likes it when the WB machine hits them once every 10 seconds, if that's even possible.
[00:06] <JAA> Yeah, and there are many people trying to archive Reddit pages through the WBM as well.
[00:07] <lennier1> I recall TwitchLeecher recently stopped working on subscriber-only VODs. I believe it said something about Twitch changing the API to prevent it. I downloaded a few with the VideoDownloadHelper Firefox extension. There's probably an easier way, but weren't really enough to videos to look for it.
[00:07] <Gallifrey> Oh really? And there was me thinking the constant 429 errors were my fault!
[00:08] <Gallifrey> Oh and Ryz, I never get close to the 12-15 pages/min limit, so no need to log in.
[00:10] <JAA> Yeah, the WBM backend only has so many IPs, and your save requests get mixed with everyone else's.
[00:14] *** britmob has quit IRC (Read error: Connection reset by peer)
[00:14] *** britmob has joined #archiveteam-bs
[00:15] <Gallifrey> Then I'm guessing the sudden uptick in 429 errors was from someone else deploying a script to archive Reddit, probably more aggressively than mine.
[00:16] *** benjinsmi has joined #archiveteam-bs
[00:17] <Gallifrey> Which seems more than likely, considering it was done on the last night before the Monday Banwave.
[00:19] *** benjins has quit IRC (Ping timeout: 265 seconds)
[00:26] *** OrIdow6 has quit IRC (Ping timeout: 265 seconds)
[00:31] <Gallifrey> Do you think it's worth archiving some of these comment threads again but with ?limit=500 added to the end of the URL?
[00:31] <Gallifrey> Obviously for the larger threads only. That way we get 500 comments instead of 200.
[00:34] <JAA> How many threads are we talking about here?
[00:38] <Gallifrey> At a guess, maybe 2000 or so per subreddit? How many subs has ArchiveBot crawled so far?
[00:41] <JAA> Quick log scan says around 1k https://old.reddit.com/r/* jobs since 2018 (which is when "old" started to be a thing).
[00:44] <Gallifrey> This is perhaps a bigger undertaking than I imagined.
[00:45] <JAA> Well, Reddit has roughly 1 billion threads.
[00:46] *** OrIdow6 has joined #archiveteam-bs
[00:46] <JAA> It doesn't make sense to do large-scale archiving of it through ArchiveBot.
[00:46] <JAA> That's why I mentioned #shreddit earlier.
[00:47] <Gallifrey> Very true. Either way, some of the at-risk subs produce a couple of dozen new threads per day, so about a 1k per month, which makes archiving them a higher priority.
[00:48] <Gallifrey> I'll remind myself to issue the request to AB in a couple of weeks then
[00:50] <Gallifrey> So how would I do it when the time comes - ask someone in the #archivebot channel? Would it help if I came with a pre-prepared list of URLs?
[00:56] <Gallifrey> I'm conscious to avoid archiving duplicates.
[00:56] *** britmob has quit IRC (Read error: Connection reset by peer)
[00:56] *** britmob has joined #archiveteam-bs
[00:57] <JAA> Yeah, #archivebot would be best. Ping me (i.e. mention my nick) so it doesn't get drowned in the bot noise there.
[00:57] <JAA> And yes, a list would be ideal.
[01:01] *** pew has quit IRC (Ping timeout: 265 seconds)
[01:02] <Gallifrey> Thanks JAA. I'll have my script scrape the Reddit API then check to see where the archived URLs stop, but not archive them myself, then upload the rest of a Pastebin document or something.
[01:08] <JAA> Sounds good.
[01:10] <Gallifrey> There are some even smaller tasks I think my script is better equipped to handle, like daily snapshots of subreddit front pages, Twitter tags, etc. I'll now be able to focus more on that.
[01:12] <Gallifrey> Anyway, it's getting late. But thanks for your help with this. Goodnight!
[01:13] *** pew has joined #archiveteam-bs
[01:18] <JAA> Repeated archival of certain things is something various people here have mentioned in the past, but we don't have anything for it really.
[01:18] <JAA> Happy to help, good night.
[01:20] <maxfan8> JAA: I'm thinking that the current script that I have could be run every year or so (along with the dedup) – once the unarchived tweets per gov account reaches a certain threshold
[01:21] <maxfan8> Idk really – just something that I was considering if this turns out well
[01:21] <JAA> maxfan8: Sure, but this is something different. I mean things like "grab https://example.org/ every 72 hours".
[01:21] <maxfan8> Ah ok
[01:27] *** atphoenix has quit IRC (Read error: Connection reset by peer)
[01:27] *** atphoenix has joined #archiveteam-bs
[01:48] *** notroot has joined #archiveteam-bs
[02:13] *** HP_Archiv has joined #archiveteam-bs
[02:21] *** HP_Archiv has quit IRC (Quit: Leaving)
[03:03] *** DogsRNice has quit IRC (Read error: Connection reset by peer)
[03:20] *** qw3rty_ has joined #archiveteam-bs
[03:27] *** qw3rty has quit IRC (Read error: Operation timed out)
[03:30] *** atphoenix has quit IRC (Ping timeout: 265 seconds)
[03:30] *** atphoenix has joined #archiveteam-bs
[04:31] *** britmob has quit IRC (Read error: Connection reset by peer)
[04:32] *** britmob has joined #archiveteam-bs
[04:33] *** benjinss has joined #archiveteam-bs
[04:35] *** benjinsmi has quit IRC (Ping timeout: 265 seconds)
[05:49] *** Lord_Nigh has quit IRC (Quit: ZNC - http://znc.in)
[05:51] *** Lord_Nigh has joined #archiveteam-bs
[06:01] *** kyledrake has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** ndiddy has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** sivoais has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** Pixi` has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** scorche has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** achip has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** Somebody2 has quit IRC (hub.efnet.us irc.Prison.NET)
[06:01] *** riking_ has quit IRC (Read error: Connection reset by peer)
[06:01] *** DrasticAc has quit IRC (Read error: Connection reset by peer)
[06:01] *** pnJay has quit IRC (Read error: Connection reset by peer)
[06:01] *** diggan has quit IRC (Read error: Connection reset by peer)
[06:01] *** horkermon has quit IRC (Read error: Connection reset by peer)
[06:01] *** maxfan8_ has joined #archiveteam-bs
[06:03] *** Maylay has quit IRC (Read error: Operation timed out)
[06:03] *** Maylay has joined #archiveteam-bs
[06:03] *** diggan has joined #archiveteam-bs
[06:03] *** DrasticAc has joined #archiveteam-bs
[06:03] *** pnJay has joined #archiveteam-bs
[06:03] *** horkermon has joined #archiveteam-bs
[06:03] *** riking_ has joined #archiveteam-bs
[06:04] *** maxfan8_ has quit IRC (Client Quit)
[06:04] *** maxfan8_ has joined #archiveteam-bs
[06:05] *** maxfan8_ has quit IRC (Client Quit)
[06:06] *** maxfan8_ has joined #archiveteam-bs
[06:06] *** maxfan8 has quit IRC (Ping timeout: 610 seconds)
[06:06] *** maxfan8_ has quit IRC (Client Quit)
[06:06] *** maxfan8 has joined #archiveteam-bs
[06:17] *** Raccoon has joined #archiveteam-bs
[06:18] *** kyledrake has joined #archiveteam-bs
[06:18] *** ndiddy has joined #archiveteam-bs
[06:18] *** sivoais has joined #archiveteam-bs
[06:18] *** Pixi` has joined #archiveteam-bs
[06:18] *** scorche has joined #archiveteam-bs
[06:18] *** achip has joined #archiveteam-bs
[06:18] *** Somebody2 has joined #archiveteam-bs
[06:25] *** ranma_ has joined #archiveteam-bs
[06:25] *** Stilett0 has joined #archiveteam-bs
[06:27] *** prq has quit IRC (Read error: Operation timed out)
[06:27] *** superkuh has quit IRC (Write error: Broken pipe)
[06:27] *** underscor has quit IRC (Write error: Broken pipe)
[06:27] *** robogoat has quit IRC (Write error: Broken pipe)
[06:27] *** underscor has joined #archiveteam-bs
[06:27] *** mtntmnky_ has quit IRC (Read error: Operation timed out)
[06:27] *** nyany has quit IRC (Read error: Operation timed out)
[06:28] *** jrwr has quit IRC (Ping timeout: 260 seconds)
[06:28] *** SynMonger has quit IRC (Read error: Operation timed out)
[06:28] *** SynMonger has joined #archiveteam-bs
[06:29] *** DigiDigi has quit IRC (Read error: Operation timed out)
[06:29] *** phirephl- has quit IRC (Ping timeout: 376 seconds)
[06:29] *** Stiletto has quit IRC (Ping timeout: 376 seconds)
[06:30] *** pie_[bnc] has quit IRC (Read error: Operation timed out)
[06:30] *** svchfoo1 has quit IRC (Read error: Operation timed out)
[06:30] *** Igloo has quit IRC (Read error: Operation timed out)
[06:30] *** ranma has quit IRC (Write error: Broken pipe)
[06:30] *** Larsenv has quit IRC (Read error: Operation timed out)
[06:30] *** sknebel has quit IRC (Read error: Operation timed out)
[06:30] *** pie_ has joined #archiveteam-bs
[06:30] *** sknebel has joined #archiveteam-bs
[06:30] *** Darkstar has quit IRC (Read error: Operation timed out)
[06:31] *** superkuh has joined #archiveteam-bs
[06:31] *** chfoo has quit IRC (Read error: Operation timed out)
[06:31] *** Iglooop1 has quit IRC (Read error: Operation timed out)
[06:31] *** atphoenix has quit IRC (Read error: Operation timed out)
[06:31] *** chfoo has joined #archiveteam-bs
[06:31] *** godane has quit IRC (Read error: Operation timed out)
[06:32] *** atphoenix has joined #archiveteam-bs
[06:32] *** robogoat has joined #archiveteam-bs
[06:33] *** yano_ has joined #archiveteam-bs
[06:34] *** phirephly has joined #archiveteam-bs
[06:35] *** godane has joined #archiveteam-bs
[06:35] *** Larsenv has joined #archiveteam-bs
[06:37] *** Frogging has quit IRC (Ping timeout: 298 seconds)
[06:37] *** simon816 has quit IRC (Ping timeout: 298 seconds)
[06:37] *** jodizzle_ has joined #archiveteam-bs
[06:39] *** jodizzle has quit IRC (Ping timeout: 312 seconds)
[06:39] *** _niklas_ has quit IRC (Ping timeout: 312 seconds)
[06:39] *** pikami_ has joined #archiveteam-bs
[06:39] *** Selavii has joined #archiveteam-bs
[06:39] *** SynMonger has quit IRC (Read error: Connection reset by peer)
[06:39] *** Darkstar has joined #archiveteam-bs
[06:39] *** Frogging has joined #archiveteam-bs
[06:39] *** JAA_ has joined #archiveteam-bs
[06:39] *** AlsoJAA sets mode: +o JAA_
[06:39] *** yano has quit IRC (Read error: Connection reset by peer)
[06:39] *** kiska has quit IRC (Ping timeout: 257 seconds)
[06:39] *** cm has quit IRC (Ping timeout: 257 seconds)
[06:39] *** pikami has quit IRC (Ping timeout: 257 seconds)
[06:39] *** swebb has quit IRC (Ping timeout: 257 seconds)
[06:39] *** bleb has joined #archiveteam-bs
[06:39] *** simon816 has joined #archiveteam-bs
[06:39] *** bleb is now known as cm
[06:39] *** kiska has joined #archiveteam-bs
[06:40] *** JAA has quit IRC (Read error: Connection reset by peer)
[06:40] *** Selavi has quit IRC (Read error: Connection reset by peer)
[06:40] *** Selavii is now known as Selavi
[06:41] *** swebb has joined #archiveteam-bs
[06:41] *** SynMonger has joined #archiveteam-bs
[06:41] *** _niklas has joined #archiveteam-bs
[06:46] *** nyany has joined #archiveteam-bs
[06:46] *** DigiDigi has joined #archiveteam-bs
[06:46] *** prq has joined #archiveteam-bs
[06:46] *** Iglooop1 has joined #archiveteam-bs
[06:47] *** svchfoo1 has joined #archiveteam-bs
[06:47] *** Craigle has quit IRC (Quit: The Lounge - https://thelounge.chat)
[06:47] *** svchfoo3 sets mode: +o svchfoo1
[06:48] *** jrwr has joined #archiveteam-bs
[06:49] *** Craigle has joined #archiveteam-bs
[06:51] *** Igloo has joined #archiveteam-bs
[06:54] *** mtntmnky_ has joined #archiveteam-bs
[08:10] *** BlueMaxim has joined #archiveteam-bs
[08:10] *** BlueMax has quit IRC (Read error: Connection reset by peer)
[08:10] *** DLoader_ has joined #archiveteam-bs
[08:12] *** Selavii has joined #archiveteam-bs
[08:13] *** jmtd has joined #archiveteam-bs
[08:14] *** ablabiX has joined #archiveteam-bs
[08:15] *** thejsa_ has joined #archiveteam-bs
[08:16] *** closure_ has joined #archiveteam-bs
[08:16] *** sirvy_ has joined #archiveteam-bs
[08:19] *** BartoCH has quit IRC (Ping timeout: 615 seconds)
[08:19] *** sirvy has quit IRC (Ping timeout: 615 seconds)
[08:19] *** amelia386 has quit IRC (Read error: Connection timed out)
[08:19] *** picklefac has quit IRC (Read error: Connection timed out)
[08:19] *** Vito` has quit IRC (Read error: Connection timed out)
[08:20] *** Selavi has quit IRC (Ping timeout: 615 seconds)
[08:20] *** c0mpass has quit IRC (Ping timeout: 1229 seconds)
[08:20] *** tchaypo_ has quit IRC (Ping timeout: 1229 seconds)
[08:20] *** Selavii is now known as Selavi
[08:20] *** BartoCH has joined #archiveteam-bs
[08:20] *** DrasticAc has quit IRC (Ping timeout: 1230 seconds)
[08:20] *** Kaz has quit IRC (Ping timeout: 1230 seconds)
[08:20] *** lenary has quit IRC (Ping timeout: 1230 seconds)
[08:20] *** closure has quit IRC (Ping timeout: 745 seconds)
[08:21] *** horkermon has quit IRC (Ping timeout: 1230 seconds)
[08:21] *** diggan has quit IRC (Ping timeout: 1230 seconds)
[08:21] *** pnJay has quit IRC (Ping timeout: 1230 seconds)
[08:21] *** Xibalba has quit IRC (Ping timeout: 745 seconds)
[08:21] *** DLoader has quit IRC (Ping timeout: 745 seconds)
[08:21] *** ablabiX is now known as Xibalba
[08:21] *** Jon| has quit IRC (Ping timeout: 745 seconds)
[08:21] *** DLoader_ is now known as DLoader
[08:22] *** Vito` has joined #archiveteam-bs
[08:22] *** starlord has quit IRC (Ping timeout: 1230 seconds)
[08:22] *** thejsa has quit IRC (Ping timeout: 745 seconds)
[08:23] *** justcool3 has quit IRC (Ping timeout: 1230 seconds)
[08:24] *** Ctrl-S___ has quit IRC (Ping timeout: 1230 seconds)
[08:24] *** mattl has quit IRC (Ping timeout: 1230 seconds)
[08:24] *** fallenoak has quit IRC (Ping timeout: 1230 seconds)
[08:25] *** picklefac has joined #archiveteam-bs
[08:25] *** HCross has quit IRC (Ping timeout: 1230 seconds)
[08:25] *** jesse-s has quit IRC (Ping timeout: 1230 seconds)
[08:26] *** riking_ has quit IRC (Read error: Connection timed out)
[08:27] *** JSharp___ has quit IRC (Read error: Connection timed out)
[08:27] *** ThisAsYou has quit IRC (Read error: Connection timed out)
[08:27] *** hook54321 has quit IRC (Read error: Connection timed out)
[08:27] *** revi has quit IRC (Read error: Connection timed out)
[08:27] *** abartov__ has quit IRC (Read error: Connection timed out)
[08:30] *** Kaz has joined #archiveteam-bs
[08:33] *** starlord has joined #archiveteam-bs
[08:38] *** amelia386 has joined #archiveteam-bs
[08:42] *** pnJay has joined #archiveteam-bs
[08:44] *** pnJay has quit IRC (Read error: Connection reset by peer)
[08:44] *** phirephl- has joined #archiveteam-bs
[08:45] *** phirephly has quit IRC (Read error: Operation timed out)
[08:46] *** c0mpass has joined #archiveteam-bs
[08:46] *** fallenoak has joined #archiveteam-bs
[08:46] *** jesse-s has joined #archiveteam-bs
[08:46] *** abartov__ has joined #archiveteam-bs
[08:46] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[08:46] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[08:46] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[08:46] *** HCross has joined #archiveteam-bs
[08:48] *** Pixi has joined #archiveteam-bs
[08:49] *** SynMonger has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** superkuh has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** kyledrake has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** ndiddy has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** sivoais has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** Pixi` has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** scorche has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** achip has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** Somebody2 has quit IRC (hub.efnet.us irc.Prison.NET)
[08:49] *** amelia386 has quit IRC (Read error: Connection reset by peer)
[08:49] *** starlord has quit IRC (Read error: Connection reset by peer)
[08:49] *** picklefac has quit IRC (Read error: Connection reset by peer)
[08:49] *** Vito` has quit IRC (Read error: Connection reset by peer)
[08:49] *** abartov__ has quit IRC (Read error: Connection reset by peer)
[08:49] *** HCross has quit IRC (Write error: Connection reset by peer)
[08:55] *** riking_ has joined #archiveteam-bs
[08:56] *** riking_ has quit IRC (Read error: Connection reset by peer)
[08:57] *** ThisAsYou has joined #archiveteam-bs
[08:58] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[08:58] *** sivoais_ has joined #archiveteam-bs
[09:02] *** hook54321 has joined #archiveteam-bs
[09:02] *** jesse-s has joined #archiveteam-bs
[09:03] *** c0mpass has joined #archiveteam-bs
[09:03] *** fallenoak has joined #archiveteam-bs
[09:07] *** mattl has joined #archiveteam-bs
[09:07] *** justcool3 has joined #archiveteam-bs
[09:07] *** Ctrl-S___ has joined #archiveteam-bs
[09:07] *** tchaypo_ has joined #archiveteam-bs
[09:08] *** HCross has joined #archiveteam-bs
[09:09] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[09:09] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[09:09] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[09:09] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[09:09] *** HCross has quit IRC (Read error: Connection reset by peer)
[09:10] *** hook54321 has quit IRC ()
[09:10] *** ThisAsYou has joined #archiveteam-bs
[09:17] *** revi has joined #archiveteam-bs
[09:17] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[09:21] *** hook54321 has joined #archiveteam-bs
[09:22] *** HCross has joined #archiveteam-bs
[09:22] *** horkermon has joined #archiveteam-bs
[09:23] *** diggan has joined #archiveteam-bs
[09:23] *** pnJay has joined #archiveteam-bs
[09:24] *** ThisAsYou has joined #archiveteam-bs
[09:26] *** pnJay has quit IRC (Read error: Connection reset by peer)
[09:26] *** diggan has quit IRC (Read error: Connection reset by peer)
[09:26] *** horkermon has quit IRC (Read error: Connection reset by peer)
[09:27] *** amelia386 has joined #archiveteam-bs
[09:28] *** picklefac has joined #archiveteam-bs
[09:28] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[09:28] *** fallenoak has joined #archiveteam-bs
[09:28] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[09:29] *** lenary has joined #archiveteam-bs
[09:30] *** HCross has quit IRC (Read error: Connection reset by peer)
[09:31] *** hook54321 has quit IRC ()
[09:33] *** hook54321 has joined #archiveteam-bs
[09:33] *** hook54321 has quit IRC (Client Quit)
[09:33] *** c0mpass has joined #archiveteam-bs
[09:34] *** ThisAsYou has joined #archiveteam-bs
[09:35] *** fallenoak has joined #archiveteam-bs
[09:36] *** HCross has joined #archiveteam-bs
[09:37] *** HCross has quit IRC (Read error: Connection reset by peer)
[09:37] *** riking_ has joined #archiveteam-bs
[09:38] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[09:39] *** tchaypo_ has joined #archiveteam-bs
[09:39] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[09:39] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[09:39] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[09:42] *** SynMonger has joined #archiveteam-bs
[09:42] *** superkuh has joined #archiveteam-bs
[09:42] *** kyledrake has joined #archiveteam-bs
[09:42] *** scorche has joined #archiveteam-bs
[09:42] *** achip has joined #archiveteam-bs
[09:42] *** Somebody2 has joined #archiveteam-bs
[09:42] *** mattl has quit IRC (Read error: Connection reset by peer)
[09:42] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[09:42] *** picklefac has quit IRC (Read error: Connection reset by peer)
[09:42] *** amelia386 has quit IRC (Read error: Connection reset by peer)
[09:42] *** revi has quit IRC (Read error: Connection reset by peer)
[09:42] *** Ctrl-S___ has quit IRC (Write error: Connection reset by peer)
[09:43] *** abartov__ has joined #archiveteam-bs
[09:45] *** tchaypo_ has joined #archiveteam-bs
[09:45] *** horkermon has joined #archiveteam-bs
[09:45] *** c0mpass has joined #archiveteam-bs
[09:46] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[09:46] *** HCross has joined #archiveteam-bs
[09:46] *** pnJay has joined #archiveteam-bs
[09:46] *** pnJay has quit IRC (Read error: Connection reset by peer)
[09:46] *** horkermon has quit IRC (Read error: Connection reset by peer)
[09:46] *** riking_ has quit IRC (Read error: Connection reset by peer)
[09:46] *** lenary has quit IRC (Read error: Connection timed out)
[09:52] *** jesse-s has joined #archiveteam-bs
[09:52] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[09:52] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[09:53] *** riking_ has joined #archiveteam-bs
[09:54] *** pnJay has joined #archiveteam-bs
[09:54] *** pnJay has quit IRC (Read error: Connection reset by peer)
[09:57] *** tchaypo_ has joined #archiveteam-bs
[09:58] *** c0mpass has joined #archiveteam-bs
[09:59] *** jesse-s has joined #archiveteam-bs
[10:00] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[10:00] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[10:00] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[10:00] *** horkermon has joined #archiveteam-bs
[10:01] *** pnJay has joined #archiveteam-bs
[10:07] *** pnJay has quit IRC (Read error: Connection reset by peer)
[10:07] *** horkermon has quit IRC (Read error: Connection reset by peer)
[10:07] *** riking_ has quit IRC (Read error: Connection reset by peer)
[10:12] *** tchaypo_ has joined #archiveteam-bs
[10:13] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[10:19] *** c0mpass has joined #archiveteam-bs
[10:20] *** tchaypo_ has joined #archiveteam-bs
[10:20] *** ThisAsYou has joined #archiveteam-bs
[10:20] *** jesse-s has joined #archiveteam-bs
[10:21] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[10:21] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[10:27] *** riking_ has joined #archiveteam-bs
[10:27] *** riking_ has quit IRC (Read error: Connection reset by peer)
[10:28] *** JSharp___ has joined #archiveteam-bs
[10:32] *** JSharp___ has quit IRC (Read error: Connection reset by peer)
[10:35] *** riking_ has joined #archiveteam-bs
[10:36] *** fallenoak has joined #archiveteam-bs
[10:37] *** jesse-s has joined #archiveteam-bs
[10:37] *** pnJay has joined #archiveteam-bs
[10:38] *** tchaypo_ has quit IRC (Read error: Connection timed out)
[10:38] *** horkermon has joined #archiveteam-bs
[10:39] *** tchaypo_ has joined #archiveteam-bs
[10:41] *** c0mpass has joined #archiveteam-bs
[11:00] *** JSharp___ has joined #archiveteam-bs
[11:11] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[11:12] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[11:12] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[11:12] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[11:20] *** horkermon has quit IRC (Read error: Connection reset by peer)
[11:20] *** pnJay has quit IRC (Read error: Connection reset by peer)
[11:20] *** riking_ has quit IRC (Read error: Connection reset by peer)
[11:22] *** jesse-s has joined #archiveteam-bs
[11:31] *** tchaypo_ has joined #archiveteam-bs
[11:31] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[11:31] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[11:38] *** jesse-s has joined #archiveteam-bs
[11:38] *** tchaypo_ has joined #archiveteam-bs
[11:40] *** horkermon has joined #archiveteam-bs
[11:40] *** pnJay has joined #archiveteam-bs
[11:40] *** riking_ has joined #archiveteam-bs
[11:40] *** riking_ has quit IRC (Read error: Connection reset by peer)
[11:40] *** pnJay has quit IRC (Read error: Connection reset by peer)
[11:40] *** horkermon has quit IRC (Read error: Connection reset by peer)
[11:42] *** c0mpass has joined #archiveteam-bs
[11:42] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[11:42] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[11:45] *** revi has joined #archiveteam-bs
[11:46] *** BlueMaxim has quit IRC (Read error: Connection reset by peer)
[11:48] *** starlord has joined #archiveteam-bs
[11:52] *** pnJay has joined #archiveteam-bs
[11:52] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[11:52] *** horkermon has joined #archiveteam-bs
[11:52] *** riking_ has joined #archiveteam-bs
[11:57] *** mattl has joined #archiveteam-bs
[11:59] *** c0mpass has joined #archiveteam-bs
[11:59] *** tchaypo_ has joined #archiveteam-bs
[12:00] *** mattl has quit IRC (Read error: Connection reset by peer)
[12:02] *** jesse-s has joined #archiveteam-bs
[12:03] *** tchaypo_ has quit IRC (Read error: Connection reset by peer)
[12:03] *** c0mpass has quit IRC (Read error: Connection reset by peer)
[12:05] *** mattl has joined #archiveteam-bs
[12:14] *** jesse-s has quit IRC (Read error: Connection reset by peer)
[12:15] *** mattl has quit IRC (Read error: Connection reset by peer)
[12:20] *** jesse-s has joined #archiveteam-bs
[12:23] *** c0mpass has joined #archiveteam-bs
[12:23] *** tchaypo_ has joined #archiveteam-bs
[12:28] *** DrasticAc has joined #archiveteam-bs
[12:30] *** diggan has joined #archiveteam-bs
[12:44] *** mattl has joined #archiveteam-bs
[13:16] *** fallenoak has joined #archiveteam-bs
[13:44] *** amelia386 has joined #archiveteam-bs
[13:46] *** mattl has quit IRC (Read error: Connection reset by peer)
[14:05] *** mattl has joined #archiveteam-bs
[14:05] *** mattl has quit IRC (Read error: Connection reset by peer)
[14:07] *** JAA_ is now known as JAA
[14:12] *** mattl has joined #archiveteam-bs
[14:16] *** mattl has quit IRC (Read error: Connection reset by peer)
[14:36] *** revi has quit IRC ()
[14:38] *** revi has joined #archiveteam-bs
[14:42] *** Vito` has joined #archiveteam-bs
[14:59] *** mattl has joined #archiveteam-bs
[15:06] *** mattl has quit IRC (Read error: Connection reset by peer)
[15:21] *** HP_Archiv has joined #archiveteam-bs
[15:27] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[15:33] *** Arcorann_ has quit IRC (Read error: Connection reset by peer)
[15:36] *** mattl has joined #archiveteam-bs
[15:36] *** mattl has quit IRC (Read error: Connection reset by peer)
[15:42] *** HP_Archiv has quit IRC (Quit: Leaving)
[15:47] *** ThisAsYou has joined #archiveteam-bs
[15:47] *** justcool3 has joined #archiveteam-bs
[15:49] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[16:00] *** mattl has joined #archiveteam-bs
[16:00] *** justcool3 has joined #archiveteam-bs
[16:02] *** Ctrl-S___ has joined #archiveteam-bs
[16:09] *** mattl has quit IRC (Read error: Connection reset by peer)
[16:13] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[16:19] *** ThisAsYou has joined #archiveteam-bs
[16:20] *** Ctrl-S___ has quit IRC (Read error: Connection reset by peer)
[16:20] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[16:26] *** justcool3 has joined #archiveteam-bs
[16:26] *** Ctrl-S___ has joined #archiveteam-bs
[16:28] *** mattl has joined #archiveteam-bs
[16:29] *** mattl has quit IRC (Read error: Connection reset by peer)
[16:29] *** Ctrl-S___ has quit IRC (Write error: Connection reset by peer)
[16:29] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[16:30] *** schbirid has joined #archiveteam-bs
[16:36] *** larryv has joined #archiveteam-bs
[16:43] *** girst has quit IRC (Quit: ZNC 1.8.0 - https://znc.in)
[16:45] *** girst has joined #archiveteam-bs
[16:47] *** mattl has joined #archiveteam-bs
[16:47] *** mattl has quit IRC (Read error: Connection reset by peer)
[16:49] *** Jon| has joined #archiveteam-bs
[16:49] *** jmtd has quit IRC (Ping timeout: 272 seconds)
[16:59] *** justcool3 has joined #archiveteam-bs
[17:00] *** mattl has joined #archiveteam-bs
[17:00] *** Ctrl-S___ has joined #archiveteam-bs
[17:08] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[17:22] *** schbirid has quit IRC (Quit: Leaving)
[17:23] *** larryv has quit IRC (Quit: larryv)
[17:23] *** Mateon1 has quit IRC (Ping timeout: 745 seconds)
[17:24] *** Mateon1 has joined #archiveteam-bs
[17:24] *** HP_Archiv has joined #archiveteam-bs
[17:38] *** ThisAsYou has joined #archiveteam-bs
[17:52] *** mattl has quit IRC (Read error: Connection reset by peer)
[17:52] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[17:52] *** JSharp___ has quit IRC (Read error: Connection reset by peer)
[17:52] *** Vito` has quit IRC (Read error: Connection reset by peer)
[17:52] *** amelia386 has quit IRC (Read error: Connection reset by peer)
[17:52] *** starlord has quit IRC (Read error: Connection reset by peer)
[17:52] *** Ctrl-S___ has quit IRC (Write error: Connection reset by peer)
[17:52] *** fallenoak has quit IRC (Write error: Connection reset by peer)
[17:52] *** jesse-s has quit IRC (Write error: Connection reset by peer)
[17:52] *** pnJay has quit IRC (Write error: Connection reset by peer)
[17:52] *** diggan has quit IRC (Write error: Connection reset by peer)
[17:52] *** DrasticAc has quit IRC (Write error: Connection reset by peer)
[17:52] *** horkermon has quit IRC (Write error: Connection reset by peer)
[17:52] *** tchaypo_ has quit IRC (Write error: Connection reset by peer)
[17:52] *** c0mpass has quit IRC (Write error: Connection reset by peer)
[17:52] *** riking_ has quit IRC (Write error: Connection reset by peer)
[17:52] *** abartov__ has quit IRC (Write error: Connection reset by peer)
[17:52] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[17:53] *** HCross has quit IRC (Write error: Connection reset by peer)
[17:54] *** Sokar has quit IRC (Ping timeout: 745 seconds)
[18:03] *** ThisAsYou has joined #archiveteam-bs
[18:04] *** hook54321 has joined #archiveteam-bs
[18:05] *** Ctrl-S___ has joined #archiveteam-bs
[18:12] *** mattl has joined #archiveteam-bs
[18:13] *** mattl has quit IRC (Read error: Connection reset by peer)
[18:13] *** hook54321 has quit IRC (Read error: Connection reset by peer)
[18:13] *** Ctrl-S___ has quit IRC (Write error: Connection reset by peer)
[18:23] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[18:23] *** horkermon has joined #archiveteam-bs
[18:23] *** horkermon has quit IRC (Read error: Connection reset by peer)
[18:24] *** HP_Archiv has quit IRC (Quit: Leaving)
[18:29] *** ThisAsYou has joined #archiveteam-bs
[18:29] *** HCross has joined #archiveteam-bs
[18:30] *** horkermon has joined #archiveteam-bs
[18:34] *** horkermon has quit IRC (Read error: Connection reset by peer)
[18:34] *** HCross has quit IRC (Read error: Connection reset by peer)
[18:34] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[18:35] *** mattl has joined #archiveteam-bs
[18:39] *** horkermon has joined #archiveteam-bs
[18:44] *** Ctrl-S___ has joined #archiveteam-bs
[18:44] *** horkermon has quit IRC (Read error: Connection reset by peer)
[18:49] *** horkermon has joined #archiveteam-bs
[18:53] *** HCross has joined #archiveteam-bs
[18:54] *** ThisAsYou has joined #archiveteam-bs
[18:55] *** HCross has quit IRC (Read error: Connection reset by peer)
[18:55] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[19:07] *** ThisAsYou has joined #archiveteam-bs
[19:07] *** HCross has joined #archiveteam-bs
[19:22] *** xit has joined #archiveteam-bs
[19:33] *** xit has quit IRC (Read error: Connection reset by peer)
[19:33] *** horkermon has quit IRC (Read error: Connection reset by peer)
[19:37] <nicolas17> I conclude getting 'all' clips from a twitch account is impossible
[19:39] *** horkermon has joined #archiveteam-bs
[19:40] *** xit has joined #archiveteam-bs
[19:40] <nicolas17> the API returns around 1000 results; I tried getting clips for the entire history of the stream, and whenever the API returned more than 200, I split the date interval into two halves and requested them separately, I saw it keep subdividing up to 3-hour intervals
[19:41] <nicolas17> now I tried a different starting interval which *still* covers the whole history, but when subdividing would cause different intervals that overlap my previous attempt... and it keeps finding new clips, some of them with hundreds of views
[19:42] <nicolas17> it feels like any change I make keeps digging more data, and I never know if I got all
[20:04] *** logchfoo4 starts logging #archiveteam-bs at Thu Jul 09 20:04:14 2020
[20:04] *** logchfoo4 has joined #archiveteam-bs
[20:06] *** starlord has joined #archiveteam-bs
[20:07] *** ThisAsYou has joined #archiveteam-bs
[20:08] *** horkermon has joined #archiveteam-bs
[20:15] *** horkermon has quit IRC (Read error: Connection reset by peer)
[20:22] *** horkermon has joined #archiveteam-bs
[20:26] *** wyatt8750 is now known as wyatt8740
[20:26] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[20:26] *** xit has joined #archiveteam-bs
[20:31] *** ThisAsYou has joined #archiveteam-bs
[20:36] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[20:39] *** HCross has joined #archiveteam-bs
[20:39] *** schbirid has joined #archiveteam-bs
[20:39] *** HCross has quit IRC (Read error: Connection reset by peer)
[20:43] *** ThisAsYou has joined #archiveteam-bs
[20:43] *** hook54321 has joined #archiveteam-bs
[20:48] *** hook54321 has quit IRC (Read error: Connection reset by peer)
[20:52] *** HCross has joined #archiveteam-bs
[20:53] *** sHATNER has joined #archiveteam-bs
[20:53] *** BlueMax has joined #archiveteam-bs
[20:53] *** JSharp___ has joined #archiveteam-bs
[20:54] *** Laverne has joined #archiveteam-bs
[20:54] *** Meli has joined #archiveteam-bs
[20:54] *** brayden has joined #archiveteam-bs
[20:56] *** HCross has quit IRC (Read error: Connection reset by peer)
[20:56] *** xit has quit IRC (Read error: Connection reset by peer)
[20:56] *** horkermon has quit IRC (Read error: Connection reset by peer)
[20:57] *** justcool3 has joined #archiveteam-bs
[20:58] *** hook54321 has joined #archiveteam-bs
[21:03] *** HCross has joined #archiveteam-bs
[21:07] *** horkermon has joined #archiveteam-bs
[21:09] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[21:10] *** prq has quit IRC (Read error: Operation timed out)
[21:10] *** mtntmnky_ has quit IRC (Read error: Operation timed out)
[21:10] *** nyany has quit IRC (Write error: Broken pipe)
[21:10] *** revi has quit IRC (Ping timeout: 260 seconds)
[21:10] *** jrwr has quit IRC (Ping timeout: 260 seconds)
[21:11] *** Pixi` has joined #archiveteam-bs
[21:12] *** phirephl- has quit IRC (Read error: Operation timed out)
[21:13] *** svchfoo1 has quit IRC (Read error: Operation timed out)
[21:13] *** Igloo has quit IRC (Read error: Operation timed out)
[21:13] *** nico_32_ has joined #archiveteam-bs
[21:13] *** nico_32 has quit IRC (Read error: Operation timed out)
[21:14] *** hook54321 has quit IRC ()
[21:14] *** Pixi has quit IRC (Read error: Operation timed out)
[21:14] *** Iglooop1 has quit IRC (Read error: Operation timed out)
[21:15] *** hook54321 has joined #archiveteam-bs
[21:16] *** horkermon has quit IRC (Read error: Connection reset by peer)
[21:17] *** phirephly has joined #archiveteam-bs
[21:17] *** DigiDigi has quit IRC (Read error: Operation timed out)
[21:17] *** pew has quit IRC (Ping timeout: 265 seconds)
[21:22] *** fallenoak has joined #archiveteam-bs
[21:25] *** qw3rty__ has joined #archiveteam-bs
[21:26] *** jodizzle_ is now known as jodizzle
[21:27] *** xit has joined #archiveteam-bs
[21:27] *** asdf0101 has quit IRC (Read error: Operation timed out)
[21:27] *** asdf0101 has joined #archiveteam-bs
[21:28] *** DigiDigi has joined #archiveteam-bs
[21:29] *** nyany has joined #archiveteam-bs
[21:29] *** prq has joined #archiveteam-bs
[21:29] *** svchfoo1 has joined #archiveteam-bs
[21:29] *** svchfoo3 sets mode: +o svchfoo1
[21:30] *** Iglooop1 has joined #archiveteam-bs
[21:30] *** revi has joined #archiveteam-bs
[21:31] *** xit has quit IRC (Client Quit)
[21:32] *** qw3rty_ has quit IRC (Read error: Operation timed out)
[21:34] *** jrwr has joined #archiveteam-bs
[21:34] *** Igloo has joined #archiveteam-bs
[21:36] *** horkermon has joined #archiveteam-bs
[21:37] *** hook54321 has quit IRC ()
[21:37] *** mtntmnky_ has joined #archiveteam-bs
[21:38] *** hook54321 has joined #archiveteam-bs
[21:39] <nicolas17> I give up; I'll zip up the metadata I gathered, upload it, and stop spending time on this :p
[21:39] *** ThisAsYou has joined #archiveteam-bs
[21:40] *** ThisAsYou has quit IRC (Read error: Connection reset by peer)
[21:41] *** Darkstar has quit IRC (http://quassel-irc.org - Chat comfortably. Anywhere.)
[21:46] *** picklefac has joined #archiveteam-bs
[21:46] *** BnAboyZ has joined #archiveteam-bs
[21:49] *** hook54321 has quit IRC (Read error: Connection reset by peer)
[21:52] *** ThisAsYou has joined #archiveteam-bs
[21:55] *** chfoo has quit IRC (Read error: Operation timed out)
[21:55] <nicolas17> http://data.nicolas17.xyz/reckful/reckful-clips-meta.ndjson.gz
[21:55] *** chfoo has joined #archiveteam-bs
[21:56] *** fallenoak has quit IRC (Read error: Connection reset by peer)
[21:57] *** HCross has quit IRC (Read error: Connection reset by peer)
[22:01] *** mattl has joined #archiveteam-bs
[22:03] *** fallenoak has joined #archiveteam-bs
[22:04] *** benjinsmi has joined #archiveteam-bs
[22:07] *** benjinss has quit IRC (Ping timeout: 265 seconds)
[22:08] *** HCross has joined #archiveteam-bs
[22:08] *** Darkstar has joined #archiveteam-bs
[22:09] *** mattl has quit IRC (Read error: Connection reset by peer)
[22:09] *** justcool3 has quit IRC (Read error: Connection reset by peer)
[22:16] *** horkermon has quit IRC (Read error: Connection reset by peer)
[22:22] *** horkermon has joined #archiveteam-bs
[22:22] *** britmob_ has joined #archiveteam-bs
[22:22] *** BlueMax has quit IRC (Read error: Connection reset by peer)
[22:23] *** horkermon has quit IRC (Read error: Connection reset by peer)
[22:23] *** britmob_ has quit IRC (Remote host closed the connection)
[22:25] *** britmob_ has joined #archiveteam-bs
[22:25] *** horkermon has joined #archiveteam-bs
[22:25] *** OrIdow6 has quit IRC (Remote host closed the connection)
[22:25] *** benjinsmi has quit IRC (Ping timeout: 265 seconds)
[22:25] *** britmob has quit IRC (Ping timeout: 265 seconds)
[22:26] *** yano has joined #archiveteam-bs
[22:26] *** benjins has joined #archiveteam-bs
[22:26] *** yano_ has quit IRC (Read error: Connection reset by peer)
[22:26] *** Zerote has quit IRC (Read error: Connection reset by peer)
[22:26] *** SJon___ has quit IRC (Ping timeout: 265 seconds)
[22:27] *** Zerote has joined #archiveteam-bs
[22:27] *** OrIdow6 has joined #archiveteam-bs
[22:28] *** SJon___ has joined #archiveteam-bs
[22:28] *** mattl has joined #archiveteam-bs
[22:29] *** justcool3 has joined #archiveteam-bs
[22:36] *** hook54321 has joined #archiveteam-bs
[22:44] *** qwebirc94 has joined #archiveteam-bs
[22:46] *** G4llifrey has joined #archiveteam-bs
[22:46] *** qwebirc94 has left 
[22:47] *** qwebirc56 has joined #archiveteam-bs
[22:47] *** qwebirc56 has quit IRC (Client Quit)
[22:52] *** Gallifrey has left 
[22:52] *** Ctrl-S___ has joined #archiveteam-bs
[22:54] *** Vito` has joined #archiveteam-bs
[22:55] *** amelia386 has joined #archiveteam-bs
[22:58] *** diggan has joined #archiveteam-bs
[22:58] *** pnJay has joined #archiveteam-bs
[22:58] *** DrasticAc has joined #archiveteam-bs
[22:58] *** jesse-s has joined #archiveteam-bs
[22:58] *** abartov__ has joined #archiveteam-bs
[22:58] *** riking_ has joined #archiveteam-bs
[22:58] *** tchaypo_ has joined #archiveteam-bs
[22:59] *** c0mpass has joined #archiveteam-bs
[23:08] *** Arcorann_ has joined #archiveteam-bs
[23:28] *** pew has joined #archiveteam-bs
[23:44] *** WalkFly has quit IRC (Quit: Textual IRC Client: www.textualapp.com)
[23:46] *** hook54321 has quit IRC ()
[23:47] *** hook54321 has joined #archiveteam-bs