[00:05] *** Sk1d has quit IRC (Read error: Operation timed out) [00:07] *** Sk1d has joined #archiveteam [00:55] *** adinbied has quit IRC (Left Channel.) [00:58] *** Sk1d has quit IRC (Read error: Operation timed out) [01:03] *** Sk1d has joined #archiveteam [01:18] *** Sk1d has quit IRC (Read error: Operation timed out) [01:21] *** Sk1d has joined #archiveteam [01:21] *** adinbied has joined #archiveteam [01:26] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [01:41] *** DoYoustil has joined #archiveteam [02:22] *** pizzaiolo has quit IRC (Read error: Connection reset by peer) [02:23] *** pizzaiolo has joined #archiveteam [02:50] *** DoYoustil has quit IRC (Read error: Connection reset by peer) [02:55] *** Stilett0 has joined #archiveteam [02:56] *** Stiletto has quit IRC (Ping timeout: 252 seconds) [03:05] *** Stiletto has joined #archiveteam [03:09] *** Stilett0 has quit IRC (Read error: Operation timed out) [03:12] *** Sk1d has quit IRC (Read error: Operation timed out) [03:14] *** Sk1d has joined #archiveteam [03:21] *** Larsenv has quit IRC (Ping timeout: 360 seconds) [03:31] *** Larsenv has joined #archiveteam [04:02] *** Stilett0 has joined #archiveteam [04:02] *** Kitaru has joined #archiveteam [04:03] *** Sk1d has quit IRC (Read error: Operation timed out) [04:03] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [04:06] *** Sk1d has joined #archiveteam [04:19] *** Sk1d has quit IRC (Read error: Operation timed out) [04:23] *** Sk1d has joined #archiveteam [04:27] *** Stiletto has joined #archiveteam [04:27] *** Stilett0 has quit IRC (Read error: Operation timed out) [04:38] *** Stiletto has quit IRC (Ping timeout: 255 seconds) [04:42] *** Stiletto has joined #archiveteam [04:42] *** Larsenv has quit IRC (Ping timeout: 246 seconds) [04:45] *** pizzaiolo has quit IRC (Quit: pizzaiolo) [04:45] *** Stilett0 has joined #archiveteam [04:47] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [04:48] *** qw3rty119 has joined #archiveteam [04:51] *** Stiletto has joined #archiveteam [04:52] *** Stilett0 has quit IRC (Ping timeout: 268 seconds) [04:54] *** qw3rty118 has quit IRC (Read error: Operation timed out) [05:02] *** alex____ has quit IRC (Quit: ZZzzz) [05:04] *** Larsenv has joined #archiveteam [05:06] *** Sk1d has quit IRC (Read error: Operation timed out) [05:08] *** Sk1d has joined #archiveteam [05:13] *** Stilett0 has joined #archiveteam [05:14] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [05:16] *** Stiletto has joined #archiveteam [05:17] *** Stilett0 has quit IRC (Ping timeout: 260 seconds) [05:22] *** Sk1d has quit IRC (Read error: Operation timed out) [05:24] *** Stilett0 has joined #archiveteam [05:25] *** Sk1d has joined #archiveteam [05:27] *** Stiletto has quit IRC (Read error: Operation timed out) [05:37] *** Stiletto has joined #archiveteam [05:37] *** Sk1d has quit IRC (Read error: Operation timed out) [05:38] *** Stilett0 has quit IRC (Read error: Operation timed out) [05:42] *** Sk1d has joined #archiveteam [05:48] *** Stiletto has quit IRC (Read error: Operation timed out) [05:50] *** Stiletto has joined #archiveteam [06:07] *** Stilett0 has joined #archiveteam [06:13] *** Larsenv has quit IRC (Ping timeout: 260 seconds) [06:13] *** Stiletto has quit IRC (Read error: Operation timed out) [06:14] *** Sk1d has quit IRC (Read error: Operation timed out) [06:17] *** Sk1d has joined #archiveteam [06:18] *** Larsenv has joined #archiveteam [06:19] *** Stiletto has joined #archiveteam [06:22] *** Stilett0 has quit IRC (Ping timeout: 633 seconds) [06:30] *** Stilett0 has joined #archiveteam [06:30] *** Stiletto has quit IRC (Ping timeout: 252 seconds) [06:33] *** Stiletto has joined #archiveteam [06:34] *** Stilett0 has quit IRC (Ping timeout: 260 seconds) [06:39] *** Stilett0 has joined #archiveteam [06:41] *** Stiletto has quit IRC (Ping timeout: 268 seconds) [06:43] *** Stiletto has joined #archiveteam [06:45] *** Stilett0 has quit IRC (Read error: Operation timed out) [06:49] *** Stilett0 has joined #archiveteam [06:49] *** Stiletto has quit IRC (Read error: Operation timed out) [06:53] *** Stiletto has joined #archiveteam [06:54] *** Stilett0 has quit IRC (Ping timeout: 260 seconds) [06:57] *** Stiletto has quit IRC (Ping timeout: 246 seconds) [06:58] *** Sk1d has quit IRC (Read error: Operation timed out) [07:00] *** Stiletto has joined #archiveteam [07:01] *** Sk1d has joined #archiveteam [07:07] *** Stilett0 has joined #archiveteam [07:13] *** Stiletto has quit IRC (Read error: Operation timed out) [07:15] *** Stiletto has joined #archiveteam [07:20] *** Stilett0 has quit IRC (Read error: Operation timed out) [07:34] *** Stilett0 has joined #archiveteam [07:39] *** Stiletto has quit IRC (Read error: Operation timed out) [07:48] *** Larsenv has quit IRC (Ping timeout: 633 seconds) [08:11] *** Larsenv has joined #archiveteam [08:13] *** Stilett0 has quit IRC (Ping timeout: 252 seconds) [08:16] *** Stiletto has joined #archiveteam [08:20] *** Stiletto has quit IRC (Ping timeout: 260 seconds) [08:21] *** Sk1d has quit IRC (Read error: Operation timed out) [08:24] *** Stiletto has joined #archiveteam [08:25] *** Sk1d has joined #archiveteam [08:28] WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD [08:28] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [08:34] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [08:37] *** Stiletto has joined #archiveteam [08:38] *** astrid has quit IRC (Read error: Operation timed out) [08:42] *** Stilett0 has joined #archiveteam [08:43] *** Stiletto has quit IRC (Ping timeout: 252 seconds) [08:46] *** Stiletto has joined #archiveteam [08:50] *** Mateon1 has quit IRC (Ping timeout: 252 seconds) [08:50] *** Mateon1 has joined #archiveteam [08:52] ok, edit done. [08:52] *** Stilett0 has quit IRC (Ping timeout: 600 seconds) [08:52] *** nertzy has joined #archiveteam [08:53] *** Stilett0 has joined #archiveteam [08:55] *** Stiletto has quit IRC (Read error: Operation timed out) [08:56] *** Stiletto has joined #archiveteam [08:57] *** Stilett0 has quit IRC (Ping timeout: 264 seconds) [08:59] *** Stilett0 has joined #archiveteam [09:01] *** Stiletto has quit IRC (Ping timeout: 260 seconds) [09:07] *** Stiletto has joined #archiveteam [09:12] *** Stilett0 has quit IRC (Read error: Operation timed out) [09:12] *** Stilett0 has joined #archiveteam [09:16] *** Stiletto has quit IRC (Ping timeout: 506 seconds) [09:23] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [09:25] *** Stilett0 has quit IRC (Ping timeout: 268 seconds) [09:26] *** Stiletto has joined #archiveteam [09:29] *** Stilett0 has joined #archiveteam [09:29] *** Sk1d has quit IRC (Read error: Operation timed out) [09:32] *** Stiletto has quit IRC (Read error: Operation timed out) [09:34] *** Sk1d has joined #archiveteam [09:35] *** Kitaru has joined #archiveteam [09:44] *** Stiletto has joined #archiveteam [09:46] *** Stilett0 has quit IRC (Read error: Operation timed out) [09:48] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [09:50] *** Sk1d has quit IRC (Read error: Operation timed out) [09:53] *** Stiletto has joined #archiveteam [09:53] *** Sk1d has joined #archiveteam [09:54] *** godane has quit IRC (Read error: Operation timed out) [10:06] *** Sk1d has quit IRC (Read error: Operation timed out) [10:08] *** Sk1d has joined #archiveteam [10:12] *** Stilett0 has joined #archiveteam [10:15] *** Stiletto has quit IRC (Read error: Operation timed out) [10:20] *** Stiletto has joined #archiveteam [10:21] *** Stilett0 has quit IRC (Ping timeout: 246 seconds) [10:26] *** Stiletto has quit IRC (Ping timeout: 246 seconds) [10:27] *** Stiletto has joined #archiveteam [10:37] *** Stiletto has quit IRC (Ping timeout: 265 seconds) [10:38] *** Stiletto has joined #archiveteam [10:38] *** alex__ has joined #archiveteam [10:42] *** Stiletto has quit IRC (Ping timeout: 255 seconds) [10:44] *** BlueMax has quit IRC (Quit: Leaving) [10:44] *** Stilett0 has joined #archiveteam [10:47] *** Stiletto has joined #archiveteam [10:49] *** Sk1d has quit IRC (Read error: Operation timed out) [10:52] *** Sk1d has joined #archiveteam [10:53] *** Stilett0 has quit IRC (Read error: Operation timed out) [10:53] *** Stilett0 has joined #archiveteam [10:53] *** Stiletto has quit IRC (Read error: Operation timed out) [10:58] *** Stilett0 has quit IRC (Ping timeout: 265 seconds) [10:58] *** Stiletto has joined #archiveteam [11:02] *** Stilett0 has joined #archiveteam [11:04] *** Sk1d has quit IRC (Read error: Operation timed out) [11:07] *** Sk1d has joined #archiveteam [11:08] *** Stiletto has quit IRC (Read error: Operation timed out) [11:16] *** pizzaiolo has joined #archiveteam [11:19] *** Sk1d has quit IRC (Read error: Operation timed out) [11:23] *** Sk1d has joined #archiveteam [11:30] *** DoYoustil has joined #archiveteam [11:38] *** Sk1d has quit IRC (Read error: Operation timed out) [11:43] *** Sk1d has joined #archiveteam [12:13] *** Stiletto has joined #archiveteam [12:16] *** zhongfu has quit IRC (Quit: cya losers) [12:17] *** zhongfu has joined #archiveteam [12:17] *** Stilett0 has quit IRC (Read error: Operation timed out) [12:32] *** Larsenv has quit IRC (Read error: Operation timed out) [12:45] *** Sk1d has quit IRC (Read error: Operation timed out) [12:46] *** Larsenv has joined #archiveteam [12:47] *** Sk1d has joined #archiveteam [13:03] *** Lilgurllu has joined #archiveteam [13:03] *** Lilgurllu has quit IRC (Client Quit) [13:15] *** Stiletto has quit IRC (Ping timeout: 246 seconds) [13:18] *** Stiletto has joined #archiveteam [13:36] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [14:11] *** sep332 has joined #archiveteam [14:12] *** wp494 has quit IRC (Ping timeout: 260 seconds) [14:12] *** wp494 has joined #archiveteam [14:14] *** matthusby has quit IRC (Remote host closed the connection) [14:15] *** matthusby has joined #archiveteam [14:24] *** Guest94 has joined #archiveteam [14:27] *** sep332 has quit IRC (Read error: Operation timed out) [14:27] *** sep332 has joined #archiveteam [14:32] *** sep332 has quit IRC (Read error: Operation timed out) [14:33] *** Guest94 has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [14:35] *** sep332 has joined #archiveteam [14:35] *** Stiletto has quit IRC (Read error: Operation timed out) [14:55] *** alex____ has joined #archiveteam [15:00] *** alex__ has quit IRC (Read error: Operation timed out) [15:02] *** paul1 has joined #archiveteam [15:09] Is this the right channel to ask about crawling a site? I'm worried that they might take down https://vermontvalleyfarm.wordpress.com/ in the not-too-distant future [15:10] related: if I officially become a warrior, can the appliance crawl a site I specify? [15:17] warrior doesn't not crawl sites per se [15:17] it downloads specific URLs it is told to [15:18] paul1: i asked #archivebot to crawl the site, it's in the queue [15:36] *** Sk1d has quit IRC (Read error: Operation timed out) [15:38] *** Sk1d has joined #archiveteam [16:21] Thanks anarcat! for all answers. [16:23] I think someone needs to poke the tracker, seems down [17:21] *** matthusb_ has joined #archiveteam [17:22] *** matthusby has quit IRC (Read error: Operation timed out) [17:30] *** Sk1d has quit IRC (Read error: Operation timed out) [17:33] fyi: transfer.sh is shutting down [17:33] 30th November [17:33] *** Sk1d has joined #archiveteam [17:33] contacting the maintainer may be a possibility: [17:33] "I'm considering a few options, so with a bit of luck we could preserve transfer.sh for the future." [17:43] well that's meta [18:07] *** pizzaiolo has quit IRC (Read error: Connection reset by peer) [18:07] *** pizzaiolo has joined #archiveteam [18:12] *** Hiccup has joined #archiveteam [18:16] *** Hiccup has quit IRC (Client Quit) [18:18] *** Hiccup has joined #archiveteam [18:19] Can anyone recommend a good "record as you browse" tool? Is pywb recommenced? [18:21] I use a chrome extension that sends the URL of every page I visit to the "save page now" feature of IA [18:21] unfortunately google removed it from the store due to privacy concerns [18:22] That's the sort of idea I'm thinking of [18:22] but I'm looking for something more thorough than that [18:22] and one that will make local copies. [18:23] It'll need take the data straight from the browser, because I'm having to use a special user agent and client certificate to access the website. [18:23] and it'll need to store the http headers of each file [18:24] *** godane has joined #archiveteam [18:35] Hiccup: There are things like warcprox or webrecorder, but I'm not sure they support client certificates. That'll probably be the big issue. [18:35] what's the best WARC *viewer* these days? pywb as well? [18:36] anarcat: Not sure if it's the best, but yeah, pywb is pretty good. [18:36] wouldn't pywb just allow you to set up your browser how you want with the client certs and stuff? [18:36] rather than having to set it up through pywb itself? [18:37] I guess I'll just try out pywb [18:37] Hiccup: All these tools typically use a MITM proxy for recording the data. And that's where the client certificate can easily throw a wrench into things. But yeah, best to just try it. [18:37] Also, -bs. [18:38] ah [18:38] I'll try it. [18:38] and I guess this channel is for AT project discussions? [18:38] I'll move to bs :P [18:49] l [18:49] *** Sk1d has quit IRC (Read error: Operation timed out) [18:51] *** patrickod has quit IRC (Read error: Operation timed out) [18:52] *** Sk1d has joined #archiveteam [19:00] *** Hiccup has quit IRC (Remote host closed the connection) [19:11] *** patrickod has joined #archiveteam [19:11] *** matthusb_ has quit IRC (Remote host closed the connection) [19:12] *** matthusby has joined #archiveteam [19:33] *** Mateon1 has quit IRC (Remote host closed the connection) [19:33] *** Mateon1 has joined #archiveteam [19:35] question, anarcat: how could I figure out what the ignore rules are from that igset you ran? [19:35] (just curious) [19:39] i don't remember [19:47] *** Sk1d has quit IRC (Read error: Operation timed out) [19:49] *** Sk1d has joined #archiveteam [20:04] *** Sk1d has quit IRC (Read error: Operation timed out) [20:07] *** Sk1d has joined #archiveteam [20:24] paul1: I believe the ignoresets for archive bot are at https://github.com/ArchiveTeam/ArchiveBot/tree/master/db/ignore_patterns [20:35] *** bRick5772 has joined #archiveteam [20:54] thanks, betamax! long live betamax [20:57] it will never die!, betamax lies, his own betamax player currently being broken [20:57] 😂 [20:58] *** Kitaru has joined #archiveteam [21:00] *** Stiletto has joined #archiveteam [21:09] *** Sk1d has quit IRC (Read error: Operation timed out) [21:12] *** Sk1d has joined #archiveteam [21:14] *** BlueMax has joined #archiveteam [21:32] *** m007a83 has quit IRC (Read error: Operation timed out) [21:42] *** Stilett0 has joined #archiveteam [21:44] *** Stiletto has quit IRC (Ping timeout: 252 seconds) [21:45] *** dashcloud has joined #archiveteam [21:46] *** m007a83 has joined #archiveteam [21:51] *** Stiletto has joined #archiveteam [21:53] *** Stilett0 has quit IRC (Ping timeout: 252 seconds) [21:54] *** Sk1d has quit IRC (Read error: Operation timed out) [21:59] *** Sk1d has joined #archiveteam [22:05] *** Stilett0 has joined #archiveteam [22:05] *** Stiletto has quit IRC (Ping timeout: 265 seconds) [22:07] *** bRick5772 has quit IRC (Quit: Leaving.) [22:12] *** Sk1d has quit IRC (Read error: Operation timed out) [22:14] *** Sk1d has joined #archiveteam [22:17] *** BlueMax has quit IRC (Read error: Connection reset by peer) [22:51] *** BlueMax has joined #archiveteam [23:30] *** m007a83 has quit IRC (Read error: Operation timed out) [23:30] *** m007a83 has joined #archiveteam [23:41] *** Kitaru has quit IRC (Quit: This computer has gone to sleep)