[00:01] *** godane has quit IRC (Ping timeout: 360 seconds) [00:21] *** godane has joined #archiveteam-ot [00:57] *** Stilettoo is now known as Stiletto [01:05] *** Zerote_ has quit IRC (Ping timeout: 252 seconds) [01:09] ealgase: I've seen cable modems that are on a stale set of channels. Check the modem's stats page to figure out if something is going wrong. [01:18] docsis 3.x, especially 3.1 does a very good job on doing initial channel selection, channel hopping and spread spectrum, 2.x and lower not so much but who uses <3.0 these days anyway? [01:20] but yeah, some CMs give you up and downstream channel specific statistics, power levels, the RTT to the CMTS, corrected and uncorrectable errors, etc. [01:57] What is a good web address for testing connections from different useragents through Curl [01:58] Flashfire: what do you want the test to reveal [01:58] If they are serving different information to what they would to say chrome [01:58] and if they are valid user agents [01:59] Was looking into Game Console UserAgents as they tend to get served different content [02:04] There is no such thing as "valid" and "invalid" UAs. You can literally send anything as the UA. [02:04] (Ok, not quite, it can't contain linebreaks for example.) [02:05] our standard UA is, I believe, something like "Archiveteam, Eat Delicious Poop" [02:06] Hehe. I think it's just "ArchiveTeam" for distributed projects, and ArchiveBot has a wonderful "ArchiveTeam ArchiveBot/... not Mozilla/5.0 ..." UA. [02:06] Although I think we might want to consider switching that actually. "Mozilla/5.0 ... actually ArchiveTeam ArchiveBot/..." would cause fewer mobile page redirect issues etc. [02:07] Anyway, to get back to the question, I'd try things like sony.com, playstation.com, that sort of sites. YouTube might also be a good candidate. [02:08] "Mozilla/5.0 Is Just Pretending To Be As Cool As Archivebot/2019" [02:08] :-) [02:36] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [02:37] Flashfire: do you want the test script part, that you can point to sites that might be varying content? [02:38] If it is minimalistic because I am still workingon figuring out code [02:44] *** t3 has quit IRC (Quit: Connection closed for inactivity) [02:50] marked: stale set of channels? what do you mean? (I'm using a Netgear router if that helps) [02:54] I meant, rebooting the modem made it go faster, I thought because it picked new channels in the process. Fusl suspects I got the explanation as to why, wrong since it was Docsis 3.0. In either case you check your modem page, see if your error rate looks way to high or something else obviously wrong. [02:55] hmm [02:55] I don't wanna reboot my modem because then my server goes down [02:56] most modems has a stats webpage, you can access from the internal LAN. [03:39] Flashfire: https://gist.github.com/marked/5b71329c8149374705218c8ed920db64 [03:40] Thanks but I have no idea what to do with it [03:46] ok, added a readme, tell me if it's understandable [03:47] That makes more sense I will try it when I get home [04:26] *** dhyan_nat has joined #archiveteam-ot [05:12] *** wp494 has quit IRC (Ping timeout: 615 seconds) [05:12] *** wp494 has joined #archiveteam-ot [06:02] *** Zerote_ has joined #archiveteam-ot [06:02] *** deevious has joined #archiveteam-ot [06:02] *** m007a83 has joined #archiveteam-ot [06:11] *** Flashfloo is now known as flashback [06:17] *** dhyan_nat has quit IRC (Read error: Operation timed out) [06:42] *** BlueMax has quit IRC (Read error: Connection reset by peer) [06:44] *** dhyan_nat has joined #archiveteam-ot [06:47] *** dhyan_nat has quit IRC (Read error: Operation timed out) [07:21] *** Zerote_ has quit IRC (Ping timeout: 252 seconds) [07:34] *** Zerote_ has joined #archiveteam-ot [09:32] *** dhyan_nat has joined #archiveteam-ot [09:58] *** dhyan_nat has quit IRC (Read error: Operation timed out) [10:19] *** tapos has joined #archiveteam-ot [11:27] *** deevious has quit IRC (Quit: deevious) [12:06] *** deevious has joined #archiveteam-ot [12:36] *** Specular has joined #archiveteam-ot [12:37] have there been any users mentioning the change in ownership of MyCE.com? [12:37] Only just discovered myself today it was sold without telling even the senior admin and three sub domains are already lost (luckily a user had a backup of one). https://club.myce.com/t/old-cdfreaks-com-sites-no-longer-exist-why/404834/14 [12:38] *senior forum admin [12:38] sold on 11th of May [12:41] *** dhyan_nat has joined #archiveteam-ot [12:48] Nope [12:49] Specular: got a list of domains? [12:50] Igloo, I only visit the forums (club.myce.com) so not familiar with the rest. The OP of the linked post above has a bunch of domains that are no longer available though. I think some are concerned for the forum, although the admins don't want people to panic. [12:56] Urgh, I think the forum software is discourse [13:03] *** logchfoo3 starts logging #archiveteam-ot at Mon May 13 13:03:34 2019 [13:03] *** logchfoo3 has joined #archiveteam-ot [13:07] I guess we should start something, perhaps AB to do a surface grab? [13:08] https://club.myce.com/about 307k topics, enough for AB to handle? [13:08] *** Oddly2 has joined #archiveteam-ot [13:16] Specular: Are you able to PM one of the admins to get a list of linked sites so we can begin archival? [13:17] Can we move this to -bs please? [13:56] Huh, so Massdrop rebranded to Drop a couple weeks ago. [13:57] *** deevious has quit IRC (Quit: deevious) [13:57] The site has always been shitty and annoying to archive, but now it's even worse. Hooray. [14:11] *** Zerote_ has quit IRC (Ping timeout: 252 seconds) [14:25] *** tapos has quit IRC (Quit: Leaving) [14:31] *** deevious has joined #archiveteam-ot [14:38] *** dhyan_nat has quit IRC (Read error: Operation timed out) [14:58] *** Zerote_ has joined #archiveteam-ot [15:34] *** dhyan_nat has joined #archiveteam-ot [16:02] *** dhyan_nat has quit IRC (Read error: Operation timed out) [16:25] *** godane has quit IRC (Read error: Connection reset by peer) [16:34] *** dhyan_nat has joined #archiveteam-ot [16:37] *** JH88 has quit IRC (JH88) [16:47] *** godane has joined #archiveteam-ot [17:19] *** dhyan_nat has quit IRC (Read error: Operation timed out) [17:51] *** godane1 has joined #archiveteam-ot [17:55] *** godane has quit IRC (Ping timeout: 360 seconds) [18:03] *** alencar has joined #archiveteam-ot [18:41] *** dhyan_nat has joined #archiveteam-ot [19:10] *** killsushi has joined #archiveteam-ot [19:14] *** dhyan_nat has quit IRC (Read error: Operation timed out) [19:32] *** t3 has joined #archiveteam-ot [19:38] *** dhyan_nat has joined #archiveteam-ot [19:59] *** Specular has quit IRC (Quit: Leaving) [20:00] *** alencar has quit IRC (Quit: Page closed) [20:12] *** killsushi has quit IRC (Quit: Leaving) [20:50] *** BlueMax has joined #archiveteam-ot [21:15] *** dhyan_nat has quit IRC (Read error: Operation timed out) [21:23] *** martini has joined #archiveteam-ot [22:18] does online.net let you use the server from the 20th to the end of the month after canceling? [22:18] *** BlueMax has quit IRC (Quit: Leaving) [22:19] ah yes apparently [22:23] *** martini has quit IRC (No Reasson) [22:36] would it be a bad idea to use the wayback machine as a way to cache API responses? [23:06] *** wp494 has quit IRC (Ping timeout: 255 seconds) [23:10] *** wp494 has joined #archiveteam-ot [23:44] wow, wpull really needs to merge some pull requests [23:44] I had to manually edit two files in line with pull requests to get it working with 3.7 [23:46] Yep, working on that. It's currently blocked by my giant PR from several months ago. [23:46] ah, makes sense [23:47] (looks like I'm going to have to merge three PR's, actually) [23:50] hmm [23:50] looks like this isn't going to work [23:50] any recommendations of other software to create WARCs? [23:52] *** Zerote_ has quit IRC (Ping timeout: 252 seconds) [23:52] Define "isn't going to work"? [23:52] as in, I've been having tons of errors [23:52] I got something about a 'dict not being hashable' [23:53] Huh. I've seen that in the test suite but never in a real run. [23:54] yeah. I can't figure out how to install it on something earlier than 3.7 either, or I'd do that (which would probably have less errors) [23:55] Yes, should work fine on 3.6.