[00:00] *** BartoCH has quit IRC (Ping timeout: 615 seconds) [00:41] *** BartoCH has joined #archiveteam-bs [00:49] *** dashcloud has quit IRC (Read error: Operation timed out) [00:50] *** closure has joined #archiveteam-bs [00:53] *** dashcloud has joined #archiveteam-bs [00:59] *** Stiletto has joined #archiveteam-bs [01:00] *** closure has quit IRC (Ping timeout: 252 seconds) [01:01] *** Stilett0 has quit IRC (Ping timeout: 268 seconds) [01:01] *** closure has joined #archiveteam-bs [01:19] *** dashcloud has quit IRC (Remote host closed the connection) [01:20] *** dashcloud has joined #archiveteam-bs [01:35] *** closure has quit IRC (Read error: Connection reset by peer) [01:35] *** closure has joined #archiveteam-bs [01:37] *** atomicthu has quit IRC (Quit: No Ping reply in 180 seconds.) [01:38] *** atomicthu has joined #archiveteam-bs [02:01] *** closure has quit IRC (Ping timeout: 260 seconds) [02:01] *** closure has joined #archiveteam-bs [02:09] *** ZizzyDizz has joined #archiveteam-bs [02:35] *** closure has quit IRC (Read error: Connection reset by peer) [02:35] *** closure has joined #archiveteam-bs [03:00] *** closure has quit IRC (Ping timeout: 268 seconds) [03:02] *** closure has joined #archiveteam-bs [03:08] *** closure has quit IRC (Read error: Connection reset by peer) [03:08] *** closure_ has joined #archiveteam-bs [03:15] *** closure_ has quit IRC (Read error: Connection reset by peer) [03:17] *** closure has joined #archiveteam-bs [03:24] *** closure has quit IRC (Read error: Connection reset by peer) [03:24] *** closure has joined #archiveteam-bs [03:28] *** archodg__ has joined #archiveteam-bs [03:30] *** archodg_ has quit IRC (Ping timeout: 252 seconds) [03:30] *** odemg has quit IRC (Ping timeout: 260 seconds) [03:31] *** closure has quit IRC (Ping timeout: 252 seconds) [03:43] *** odemg has joined #archiveteam-bs [04:02] I hope we had a copy of kanye west stuff cause he deleted his instagram and twitter [04:05] https://www.inoreader.com/subscription/65818821 has some [04:06] via a twitrss feed [04:06] or https://www.inoreader.com/feed/https%3A%2F%2Ftwitrss.me%2Ftwitter_user_to_rss%2F%3Fuser%3Dkanyewest [04:07] http version goes back further in time https://www.inoreader.com/feed/http%3A%2F%2Ftwitrss.me%2Ftwitter_user_to_rss%2F%3Fuser%3Dkanyewest [04:07] Apr 2018 [04:08] er, July 2014 [04:09] https://web.archive.org/web/*/https://twitter.com/kanyewest/ seems to have quite a few snapshots [04:50] *** BartoCH has quit IRC (Ping timeout: 615 seconds) [05:08] *** BartoCH has joined #archiveteam-bs [05:20] *** BartoCH has quit IRC (Ping timeout: 615 seconds) [05:26] *** BartoCH has joined #archiveteam-bs [06:32] *** dashcloud has quit IRC (Read error: Operation timed out) [06:55] *** icedice has joined #archiveteam-bs [07:04] *** SynMonger has quit IRC (Read error: Operation timed out) [07:06] *** djsundog has quit IRC (Read error: Operation timed out) [07:09] *** SynMonger has joined #archiveteam-bs [07:11] *** djsundog has joined #archiveteam-bs [08:57] *** VerifiedJ has joined #archiveteam-bs [09:09] *** VerifiedJ has quit IRC (Read error: Connection reset by peer) [09:11] *** ZizzyDizz has quit IRC (Ping timeout: 260 seconds) [09:21] *** faolingf_ has quit IRC (Quit: Leaving) [10:32] *** VerifiedJ has joined #archiveteam-bs [12:10] *** BlueMax has quit IRC (Quit: Leaving) [15:31] *** Guestiiii has joined #archiveteam-bs [15:31] Hey [15:32] ivan: what about making a webextension that dumps the document tree and pushes it to a local server ? [15:35] i.e. http PUSH [15:35] Also, does grab-site has an api for http communication like that? [15:35] *have [15:38] you can do this with puppeteer [15:38] or with https://github.com/PromyLOPh/crocoite [15:39] link to pupeeter? [15:40] https://github.com/GoogleChrome/puppeteer [15:40] some sample code, good luck implementing all the scrolling and clicking https://gist.github.com/ivan/39751d44b9b6f8644ce8177339a2f459 [15:40] mmh [15:41] oh, but I didn't mean some headless thing [15:41] how would the web extension be an improvement over doing ctrl-s? [15:41] I just meant: you're looking at a webpage, click "bookmark" or whatever, and the extension, in addition to bookmarking it, pushes the data [15:42] but didn't you say ctrl-s doesn't work that well in firefox? [15:42] you can get an extension to fix it [15:42] also, the improvement would be the combo bookmark+ctrl-s in one click [15:42] your idea is weird to me because I save far more things than I bookmark [15:42] but I guess you could write something that does that, yes [15:42] ok, thanks [15:43] and does grab-site listen to http for sites to archive? [15:44] here's how puppeteer gets the DOM https://github.com/GoogleChrome/puppeteer/blob/07febb637c78cd59e22a15166f816d838a36e614/lib/FrameManager.js#L508-L520 [15:45] Guestiiii: there's no real control interface in grab-site except through command line operation and the control files [15:45] hah, thanks [15:45] there's a websocket but it's just for grab-site processes to report what they're doing [15:45] but that's ugly! [15:45] (the dom thing) [15:46] what's wrong with it [15:46] I don't know, I would have expected it to be 2 lines [15:46] isn't the dom just one big xml tree or something? [15:46] yes [15:47] how do you dump a tree? serialize it [15:47] well, it's text no? [15:47] the XMLSerializer is just for the doctype [15:47] the browser does the rest of the work with the .outerHTML [15:47] but that's for old pages, the relevant line is document.documentElement.outerHTML [15:47] .doctype is the at the top [15:47] ^ yea right, thought there was an else [15:48] effectively two lines of code [15:48] ah [15:48] oh, you were linking to specific lines [15:48] I think my adblocker messed that [15:49] it's a big file, if you scroll before it is ready you don't end up on the correct lines [15:49] yeah, ok [15:50] and is the js and css somehow embeddid in outerHTML or? [15:50] there is no external resources [15:50] ivan: my general rationale is I want to have a relatively easy way to keep track (and hopefully relatively good archives) of sites: but those are only ones I bookmark [15:51] also, if I were to write an extension like mentionned, I could easily make it send links, the html and other stuff; that is, I can adapt it [15:51] Meroje: what do you mean exactly? [15:51] *** schbirid has joined #archiveteam-bs [15:52] https://pinboard.in/faq/#search_scope does this for you [15:53] Yeah, but that's a company, not my own computer! [15:54] Note: I'm not trying to argue over anything, if you guys start getting weary of my arguments, tell me before I make a fool of myself! [15:54] pinboard is love [15:56] isn't it paying anyway? [15:57] *** closure has joined #archiveteam-bs [15:58] there's webrecorder too as an easy-ish solution to selfhost [16:06] using headless chrome? [17:04] *** wp494 has quit IRC (Quit: LOUD UNNECESSARY QUIT MESSAGES) [17:04] *** Stilett0 has joined #archiveteam-bs [17:06] *** wp494 has joined #archiveteam-bs [17:11] *** Stiletto has quit IRC (Read error: Operation timed out) [17:26] *** Stiletto has joined #archiveteam-bs [17:27] gtg, thanks for the infos! [17:27] *** Guestiiii has quit IRC (Remote host closed the connection) [17:29] *** Stilett0 has quit IRC (Read error: Operation timed out) [17:56] *** icedice has quit IRC (Quit: Leaving) [17:58] *** Stilett0 has joined #archiveteam-bs [17:59] *** Stiletto has quit IRC (Read error: Operation timed out) [18:33] *** Stiletto has joined #archiveteam-bs [18:36] *** Stilett0 has quit IRC (Read error: Operation timed out) [18:51] *** schbirid has quit IRC (Remote host closed the connection) [18:55] *** Mateon1 has quit IRC (Ping timeout: 252 seconds) [18:55] *** Mateon1 has joined #archiveteam-bs [20:16] *** Pixi` has quit IRC (Quit: Pixi`) [20:24] *** BlueMax has joined #archiveteam-bs [20:25] *** Pixi has joined #archiveteam-bs [22:03] https://docs.google.com/spreadsheets/d/1cU6AZnWyJu7tEuqfgLaq_kuAMBFQFtsY8rNof9fBRfk/edit [22:03] List of Wikis migrating [22:03] JAA [22:04] *** m007a83_ has joined #archiveteam-bs [22:04] Flashfire: you've pasted this in every channel but no explanation [22:04] why [22:05] *** m007a83 has quit IRC (Ping timeout: 252 seconds) [22:05] *** m007a83_ is now known as m007a83 [22:05] List of wikis migrating from wikia to fandom [22:06] they will all eventually move from wikia to fandom but they are the trial ones moving [22:06] ok [22:17] *** VerifiedJ has quit IRC (Quit: Leaving) [22:17] *** m007a83_ has joined #archiveteam-bs [22:19] *** m007a83 has quit IRC (Read error: Operation timed out) [22:34] *** BlueMax has quit IRC (Quit: Leaving) [22:50] *** Stiletto has quit IRC (Read error: Operation timed out) [22:50] *** Stilett0 has joined #archiveteam-bs [22:58] *** m007a83 has joined #archiveteam-bs [23:02] *** m007a83_ has quit IRC (Read error: Operation timed out) [23:04] *** arbin_ has quit IRC (Read error: Operation timed out) [23:04] *** Polylith_ has quit IRC (Read error: Operation timed out) [23:05] *** svchfoo3 has quit IRC (Read error: Operation timed out) [23:05] *** Polylith has joined #archiveteam-bs [23:10] *** svchfoo3 has joined #archiveteam-bs [23:11] *** svchfoo1 sets mode: +o svchfoo3 [23:14] *** Stiletto has joined #archiveteam-bs [23:16] *** Stilett0 has quit IRC (Read error: Operation timed out) [23:16] *** arbin has joined #archiveteam-bs [23:23] *** Stilett0 has joined #archiveteam-bs [23:24] *** Stiletto has quit IRC (Ping timeout: 260 seconds)