[00:00] *** Nertsy has joined #archiveteam-bs [00:02] *** Lord_Nigh has joined #archiveteam-bs [00:24] *** Nertsy has quit IRC (Ping timeout: 512 seconds) [00:25] *** Nertsy has joined #archiveteam-bs [00:31] *** Lord_Nigh has quit IRC (Read error: Connection reset by peer) [00:31] *** Lord_Nigh has joined #archiveteam-bs [00:57] *** cbb2 has joined #archiveteam-bs [00:57] *** Nertsy has quit IRC (Read error: Operation timed out) [01:05] *** Nertsy has joined #archiveteam-bs [01:06] *** Nertsy has quit IRC (Client Quit) [01:10] *** Nertsy has joined #archiveteam-bs [01:52] *** Start has quit IRC (Disconnected.) [01:53] *** Start has joined #archiveteam-bs [01:55] *** winr5r is now known as winr4r [01:59] Idea: Preemptive Web archival — have IA offer web hosting, and keep track of all changes to the files, and automatically make WARCs for each unique document... [02:03] https://www.archive-it.org/ [02:04] *** cbb2 has quit IRC (cbb2) [02:05] *** mistym has quit IRC (Remote host closed the connection) [02:07] I thought that was just crawling existing websites sort of like Archivebot does, not actually hosting and tracking changes [02:07] from what I remember from the webinar a while ago archive-it was like wayback machine as a service, you tell it what pages to crawl and curate the portal [02:08] Right, I'm talking about actual hosting à la Geocities [02:08] IAcities [02:10] a browser extension that automatically download every youtube video you watch would also be nice [02:10] web hosting with archiving built in, could be cool [02:11] Ravenloft, maybe grep through your history and youtube-dl it? [02:12] key word: automatically [02:13] well, that would be semi-automatic :P [02:13] could put it in your crontab maybe [02:18] *** primus104 has quit IRC (Leaving.) [02:19] I appreciate your input, but I said browser extension [02:20] the idea is to make something easy to install and use [02:20] for the average user [02:20] *** mistym has joined #archiveteam-bs [02:21] I know you did. I'm just tossing out ideas [02:21] ideally this would send metadata to somewhere [02:21] * winr4r hugs mistym [02:21] and when its a video not already saved there it would do it [02:21] kinda like a cache [02:21] for youtube videos [02:22] if you ever watched it, it would be there, even if it was removed [02:22] by the user [02:22] or google [02:22] Ravenloft: last i heard IA archives any youtube video mentioned on twitter [02:23] yeah, thats good [02:35] *** antonizoo has joined #archiveteam-bs [02:36] so, about archivebot... [02:36] how do I set it up such that I am in control of it [02:37] archivebot is in two parts: backend https://github.com/ArchiveTeam/ArchiveBot/blob/master/INSTALL.backend and pipeline https://github.com/ArchiveTeam/ArchiveBot/blob/master/INSTALL.pipeline [02:37] There's instructions in https://github.com/ArchiveTeam/ArchiveBot/blob/master/INSTALL.backend and https://github.com/ArchiveTeam/ArchiveBot/blob/master/INSTALL.pipeline [02:37] jinx [02:37] cool [02:38] be forewarned it's not the easiest to install [02:39] yeah, I'm starting to notice [02:39] if the installation steps have to be done exactly right [02:39] it is rather tricky, I'd start with the backend probably [02:39] maybe it would be better to have an install script? [02:39] I mean, it has to be done the same everytime, right? [02:40] if I remember correctly that is somewhere on the to do list for it [02:41] check the travis file [02:41] Ah, interesting. I've heard of travis, what does it do [02:42] https://github.com/ArchiveTeam/ArchiveBot/blob/master/.travis.yml [02:42] it's for basic testing the bot [02:43] So, it's an install script that is already engineered to work... right? [02:43] you can also use our archivebot [02:43] oh yeah, that would be better too [02:43] here are the URLs: [02:43] yeah, just use #archivebot [02:44] so... how do you submit a URL? [02:44] there's an irc bot in #archivebot [02:44] *** dashcloud has quit IRC (Read error: Operation timed out) [02:44] http://archivebot.rtfd.org/en/latest/ [02:45] well, I do need permissions to talk there, right? [02:45] it's not +m but you need voice to issue bot commands [02:46] ok. How do I get voice [02:46] you have it [02:46] oh, thanks [02:49] *** dashcloud has joined #archiveteam-bs [02:53] So whats happening with blogger? What is getting changed/going down? [02:53] Is it just the adult content takedown? [02:53] yeah adult content [02:54] but that's always kind of hard to define [02:54] yea [02:55] Luckily all of the Trix Cereal blog sites should be safe [02:55] phew [03:10] Have their been any go to settings for grabbing stuff off blogger in bulk? [03:10] I read some comments the other day about running into page request issues [03:10] and needing a high page request randomization time [03:11] they do have a pretty good rate limiter but with a randomized delay time and a distributed warrior base it shouldn't be too bad [03:16] right, my grab of newton completed \o/ [03:16] http://wat.lewiscollard.com/archive/www.newton.dep.anl.gov/ [03:17] the randomised header images broke for reasons i do not understand [03:17] otherwise it's all there [03:21] imma fix that then tgz it and that way anyone can easily set up a mirror [04:27] i'm getting direct to https://:/ when trying to go to archive.org [04:27] very odd [04:27] can't got to front page [04:28] *go to front page [04:56] *** mistym has quit IRC (Remote host closed the connection) [04:58] godane: clear your cache and reload? [04:59] thats the thing [04:59] i moved .mozilla folder and still had this problem [05:00] huh [05:02] ok that worked [05:03] i just moved it but didn't tell firefox to clear cache [05:04] *** swebb has quit IRC (Ping timeout: 319 seconds) [05:04] *** goekesmi has quit IRC (Ping timeout: 319 seconds) [05:04] *** Zebranky has quit IRC (Ping timeout: 319 seconds) [05:06] *** swebb has joined #archiveteam-bs [05:12] *** goekesmi has joined #archiveteam-bs [05:13] *** Zebranky has joined #archiveteam-bs [05:14] *** aaaaaaaaa has quit IRC (Leaving) [05:28] *** mistym has joined #archiveteam-bs [06:09] *** sep332 has quit IRC (Read error: Operation timed out) [06:20] *** sep332 has joined #archiveteam-bs [07:22] *** Smiley has quit IRC (Remote host closed the connection) [07:22] *** Smiley has joined #archiveteam-bs [07:22] *** antomati_ has joined #archiveteam-bs [07:25] *** antomatic has quit IRC (Ping timeout: 246 seconds) [07:33] *** antonizoo has quit IRC (Quit: Konversation terminated!) [07:40] *** SmileyG has joined #archiveteam-bs [07:42] *** winr5r has joined #archiveteam-bs [07:43] *** lrkj has joined #archiveteam-bs [07:44] *** musalbas has joined #archiveteam-bs [07:44] *** Smiley has quit IRC (hub.efnet.us irc.colosolutions.net) [07:44] *** Zebranky has quit IRC (hub.efnet.us irc.colosolutions.net) [07:45] *** balrog_ has joined #archiveteam-bs [07:46] *** Arkiver2 has joined #archiveteam-bs [07:47] *** joepie91_ has joined #archiveteam-bs [07:47] *** primus104 has joined #archiveteam-bs [07:48] *** nico_32_ has joined #archiveteam-bs [07:49] *** RedType_ has joined #archiveteam-bs [07:51] *** Zebranky_ has joined #archiveteam-bs [07:51] *** dashcloud has quit IRC (Read error: Operation timed out) [07:57] *** dashcloud has joined #archiveteam-bs [08:05] *** GLaDOS has quit IRC (hub.se efnet.port80.se) [08:05] *** Famicoman has quit IRC (hub.se efnet.port80.se) [08:05] *** RedType has quit IRC (hub.se efnet.port80.se) [08:05] *** lysobit has quit IRC (hub.se efnet.port80.se) [08:05] *** arkiver has quit IRC (hub.se efnet.port80.se) [08:05] *** lrkj_ has quit IRC (hub.se efnet.port80.se) [08:05] *** balrog has quit IRC (hub.se efnet.port80.se) [08:05] *** nico_32 has quit IRC (hub.se efnet.port80.se) [08:05] *** Sue_ has quit IRC (hub.se efnet.port80.se) [08:05] *** winr4r has quit IRC (hub.se efnet.port80.se) [08:05] *** joepie91 has quit IRC (hub.se efnet.port80.se) [08:05] *** Muad-Dib has quit IRC (hub.se efnet.port80.se) [08:05] *** danneh_ has quit IRC (hub.se efnet.port80.se) [08:05] *** deathy has quit IRC (hub.se efnet.port80.se) [08:13] *** mistym has quit IRC (Remote host closed the connection) [08:17] *** chazchaz has quit IRC (Ping timeout: 369 seconds) [08:20] *** musalbas is now known as lysobit [08:20] *** balrog_ is now known as balrog [08:28] *** GLaDOS has joined #archiveteam-bs [08:29] *** Sue_ has joined #archiveteam-bs [08:40] *** antomati_ is now known as antomatic [08:53] *** Famicoman has joined #archiveteam-bs [08:56] *** primus104 has quit IRC (Leaving.) [08:59] *** sep332 has quit IRC (Read error: Operation timed out) [09:22] *** sep332 has joined #archiveteam-bs [09:46] *** Famicoman has quit IRC (hub.se efnet.port80.se) [09:46] *** Sue_ has quit IRC (hub.se efnet.port80.se) [09:46] *** GLaDOS has quit IRC (hub.se efnet.port80.se) [10:00] *** Famicoma1 has joined #archiveteam-bs [10:02] *** deathy has joined #archiveteam-bs [10:18] *** schbirid has joined #archiveteam-bs [11:06] *** Arkiver2 is now known as arkiver [11:17] *** nico_32_ is now known as nico_32 [11:20] *** BlueMaxim has quit IRC (Read error: Connection reset by peer) [11:35] what a bunch of total cocks https://www.youtube.com/watch?v=mPUjvP-4Xaw [11:38] *** winr5r is now known as winr4r [11:43] *** primus104 has joined #archiveteam-bs [11:52] *** primus105 has joined #archiveteam-bs [11:58] *** Sue_ has joined #archiveteam-bs [12:00] *** primus104 has quit IRC (Read error: Operation timed out) [13:10] *** WubTheCap has joined #archiveteam-bs [13:20] *** xtr-201 has quit IRC (Ping timeout: 370 seconds) [13:52] *** garyrh has quit IRC (Remote host closed the connection) [13:56] *** sankin has joined #archiveteam-bs [14:09] we maybe getting pri open source podcast [14:18] *** ocwn87 has joined #archiveteam-bs [14:18] *** ocwn87 is now known as Spiritt [14:18] hey, does someone know super secret toolz to find historic whois information of obscure subdomains? [14:37] *** danneh_ has joined #archiveteam-bs [14:41] *** dashcloud has quit IRC (Read error: Operation timed out) [14:41] *** garyrh has joined #archiveteam-bs [14:48] *** dashcloud has joined #archiveteam-bs [14:53] Spiritt: subdomains don't have whois information [14:59] ah, doh [14:59] i was panicking a bit because i found some repos on a subdomain of a gov agency, grabbed half of them and then the server was gone =) [15:46] *** Muad-Dib has joined #archiveteam-bs [16:08] *** mistym has joined #archiveteam-bs [16:08] *** mistym has quit IRC (Remote host closed the connection) [16:13] Spiritt: for DNS there's https://www.deepmagic.com/ but it's down at the moment [16:13] *** xtr-201 has joined #archiveteam-bs [16:16] *** primus105 has quit IRC (Leaving.) [16:25] *** mistym has joined #archiveteam-bs [16:34] *** Nertsy has quit IRC (Ping timeout: 512 seconds) [16:35] *** Nertsy has joined #archiveteam-bs [16:43] *** aaaaaaaaa has joined #archiveteam-bs [17:19] *** chazchaz has joined #archiveteam-bs [17:21] *** primus104 has joined #archiveteam-bs [17:36] *** mistym has quit IRC (Remote host closed the connection) [17:51] *** mistym has joined #archiveteam-bs [18:09] *** xtr-201 has quit IRC (Ping timeout: 370 seconds) [18:10] *** chris_ has joined #archiveteam-bs [18:10] *** chris_ is now known as wacky [18:14] *** Spiritt has quit IRC (Quit: Leaving) [19:06] *** xmc sets mode: +o swebb [19:06] *** swebb sets mode: +o balrog [19:14] *** garyrh_ has quit IRC (Quit: Leaving) [19:20] *** garyrh_ has joined #archiveteam-bs [20:45] *** miljo has quit IRC (leaving) [21:03] *** xtr-201 has joined #archiveteam-bs [21:10] *** chfoo has quit IRC (Quit: chfoo) [21:15] *** chfoo has joined #archiveteam-bs [21:16] *** chfoo has quit IRC (Remote host closed the connection) [21:18] *** chfoo has joined #archiveteam-bs [21:21] *** mistym has quit IRC (Remote host closed the connection) [21:29] *** BlueMaxim has joined #archiveteam-bs [21:37] *** schbirid has quit IRC (Leaving) [21:44] *** mistym has joined #archiveteam-bs [21:48] *** sankin has quit IRC (Leaving.) [22:03] *** Ravenloft has quit IRC (Read error: Operation timed out) [22:06] *** SN4T14 has quit IRC (Quit: Leaving) [22:09] *** pwnsrv has joined #archiveteam-bs [22:18] https://www.reddit.com/r/talesfromtechsupport/comments/2x4iz6/so_secure_you_cant_read_it/coxozj3?context=3 [22:18] some maybe interesting links [22:20] *** mistym has quit IRC (Remote host closed the connection) [22:36] *** mistym has joined #archiveteam-bs [22:45] *** primus_ has quit IRC (Read error: Operation timed out) [22:47] *** primus_ has joined #archiveteam-bs [23:02] *** rejon has joined #archiveteam-bs