[00:00] *** Marc| has joined #archiveteam [00:02] *** Marc| has quit IRC (Client Quit) [00:03] *** Marc| has joined #archiveteam [00:04] *** Specular is now known as Spring [00:06] *** Marc| has quit IRC (Client Quit) [00:10] *** Marc| has joined #archiveteam [00:14] *** RealMarc has quit IRC (Quit: ZNC - http://znc.in) [00:14] *** Marc| is now known as RealMarc [00:16] *** w0rp has quit IRC (Ping timeout: 370 seconds) [00:25] *** w0rp has joined #archiveteam [00:34] *** kyan has quit IRC (Quit: Leaving) [00:37] *** abartov has joined #archiveteam [00:46] *** Riviera has joined #archiveteam [01:23] *** philpem has quit IRC (Ping timeout: 272 seconds) [01:28] *** mistym has quit IRC (Remote host closed the connection) [01:34] *** Specular has joined #archiveteam [01:37] *** yan has quit IRC (Quit: leaving) [01:39] *** Spring has quit IRC (Ping timeout: 369 seconds) [01:55] *** BlueMaxim has quit IRC (Quit: Leaving) [01:56] *** Spring has joined #archiveteam [02:02] *** Specular has quit IRC (Ping timeout: 369 seconds) [02:03] *** RedRiderX has quit IRC (Ping timeout: 335 seconds) [03:04] *** Specular has joined #archiveteam [03:09] *** Spring has quit IRC (Ping timeout: 369 seconds) [03:41] *** APerti has joined #archiveteam [04:03] *** mistym has joined #archiveteam [04:15] *** Ymgve has quit IRC () [04:17] *** aMunster has quit IRC (Remote host closed the connection) [04:31] *** BlueMaxim has joined #archiveteam [04:35] *** aMunster has joined #archiveteam [04:41] They get by [04:45] *** GLaDOS has quit IRC (Ping timeout: 272 seconds) [04:51] *** Start is now known as StartAway [04:54] *** aaaaaaaaa has quit IRC (Leaving) [04:57] *** StartAway is now known as Start [05:06] *** APerti has quit IRC (Read error: Operation timed out) [05:10] *** APerti has joined #archiveteam [05:21] *** aschmitz has quit IRC (Read error: Operation timed out) [05:23] *** kyan has joined #archiveteam [05:33] *** aschmitz has joined #archiveteam [05:34] *** Start is now known as StartAway [05:36] *** APerti has quit IRC (Read error: Operation timed out) [06:11] *** ivan- has joined #archiveteam [06:12] does anyone know anyone at github who could put ia_archiver in https://gist.github.com/robots.txt? [06:12] github.com itself whitelisted it last year https://twitter.com/textfiles/status/461768909725913088 [06:13] do we have all of github archived? [06:13] like all the repos [06:13] there may be a secret IA effort but I have no status on it [06:14] other than that possibility, no [06:29] *** BlueMaxim has quit IRC (Ping timeout: 335 seconds) [06:46] *** GLaDOS has joined #archiveteam [06:46] *** swebb sets mode: +o GLaDOS [07:20] *** __uu has quit IRC (Ping timeout: 265 seconds) [07:27] *** __uu has joined #archiveteam [07:42] *** garyrh has quit IRC (Read error: Operation timed out) [07:45] *** db48x has joined #archiveteam [07:45] do we have a script that can save ReadCube documents? [07:48] http://www.readcube.com/articles/10.1038%2Fnature14098?shared_access_token=vs1LSOS0rsyCmVthP_qHRtRgN0jAjWel9jnR3ZoTv0PvwA6rMMnycnymQk5ZOpb5pHHQVfVWU9jOczAYnpFGkA3sjga3REnxKT2iKrU56ACJmfihRZ8WJ52nwFVZyQ5c7cBH4LnCvbER2Mvz_TMzKhQT-pdhYmlVor3WYloMbPqta0iNJWtuvhKXEGu2NRv2tY68XzIChJmykYxvrqFGMg%3D%3D [07:49] *** mistym has quit IRC (Remote host closed the connection) [07:52] the main complication is that they load each page dynamically as you scroll down, and each page load has a signature which must match [07:53] a warc proxy will obviously work fine, if you drive a browser and make it scroll down [07:53] and if the webpage can generate the signature in JS, then we can do it in lua, so wget will ultimately work too [07:54] *** garyrh has joined #archiveteam [07:58] it seems like it would be more valuable to rip them to PDFs and upload them to scimag [07:59] you have to download the html before you can convert them to anything [08:00] phantomjs? [08:00] * midas1 hides [08:00] even the stylesheets and background images have this stupid signature [08:01] midas1: yea, that could be doable as well [08:02] href="http://red2-assets.readcube.com/prerendered/bdf10a06077454482f2c503cd6cb549d6b9eb018e60ee1192dd5a79a744622ae/style.css?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cDovL3JlZDItYXNzZXRzLnJlYWRjdWJlLmNvbS9wcmVyZW5kZXJlZC9iZGYxMGEwNjA3NzQ1NDQ4MmYyYzUwM2NkNmNiNTQ5ZDZiOWViMDE4ZTYwZWUxMTkyZGQ1YTc5YTc0NDYyMmFlL3N0eWxlLmNzcyIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTQyMDg0ODAwMH0sIklwQWRkcmVzcyI6eyJBV1M6U291c [08:02] media="all" rel="stylesheet" /> [08:04] it has 17 embedded fonts in it :P [08:18] I must sleep [08:18] but I'll cogitate on the matter, I'm sure [08:21] *** khaoohs__ has joined #archiveteam [08:21] *** khaoohs_ has quit IRC (Read error: Connection reset by peer) [08:39] *** ruukasu has quit IRC (Quit: WeeChat 1.0.1) [08:43] *** ruukasu has joined #archiveteam [08:45] *** schbirid has joined #archiveteam [08:57] *** primus104 has joined #archiveteam [09:05] *** BlueMaxim has joined #archiveteam [09:17] *** Fusl has quit IRC (Ping timeout: 240 seconds) [09:22] *** brayden has quit IRC (Ping timeout: 606 seconds) [09:30] *** RealMarc has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** dashcloud has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** Elegance has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** DFJustin has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** offby1 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** StartAway has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** nico_32 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** will__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** Sellyme has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** underscor has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** maltris has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** pikhq_ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** everdred has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** wacky has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** ryan__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:30] *** matthusb- has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:31] *** brayden has joined #archiveteam [09:31] *** Sellyme_ has joined #archiveteam [09:33] *** RealMarc has joined #archiveteam [09:33] *** Elegance has joined #archiveteam [09:33] *** DFJustin has joined #archiveteam [09:33] *** offby1 has joined #archiveteam [09:33] *** torvik has joined #archiveteam [09:33] *** SadDM has joined #archiveteam [09:33] *** nico_32 has joined #archiveteam [09:33] *** useretail has joined #archiveteam [09:33] *** trs80 has joined #archiveteam [09:33] *** will__ has joined #archiveteam [09:33] *** wacky has joined #archiveteam [09:33] *** maltris has joined #archiveteam [09:33] *** rduser has joined #archiveteam [09:33] *** pikhq_ has joined #archiveteam [09:33] *** everdred has joined #archiveteam [09:33] *** ryan__ has joined #archiveteam [09:33] *** matthusb- has joined #archiveteam [09:33] *** irc.shaw.ca sets mode: +oo DFJustin SadDM [09:33] *** DFJustin has quit IRC (Remote host closed the connection) [09:33] *** swebb sets mode: +o SadDM [09:37] *** RealMarc has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** Elegance has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** offby1 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** nico_32 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** will__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** maltris has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** pikhq_ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** wacky has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** everdred has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** ryan__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:37] *** matthusb- has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:40] *** RealMarc has joined #archiveteam [09:40] *** Elegance has joined #archiveteam [09:40] *** offby1 has joined #archiveteam [09:40] *** torvik has joined #archiveteam [09:40] *** SadDM has joined #archiveteam [09:40] *** nico_32 has joined #archiveteam [09:40] *** useretail has joined #archiveteam [09:40] *** trs80 has joined #archiveteam [09:40] *** will__ has joined #archiveteam [09:40] *** wacky has joined #archiveteam [09:40] *** maltris has joined #archiveteam [09:40] *** rduser has joined #archiveteam [09:40] *** pikhq_ has joined #archiveteam [09:40] *** everdred has joined #archiveteam [09:40] *** ryan__ has joined #archiveteam [09:40] *** matthusb- has joined #archiveteam [09:40] *** irc.shaw.ca sets mode: +o SadDM [09:40] *** swebb sets mode: +o SadDM [09:44] *** RealMarc has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** Elegance has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** offby1 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** torvik has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** nico_32 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** trs80 has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** will__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** maltris has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** rduser has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** pikhq_ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** wacky has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** everdred has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** ryan__ has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:44] *** matthusb- has quit IRC (ircd.shaw.ca irc.shaw.ca) [09:49] *** maltris_ has joined #archiveteam [09:50] *** Marc- has joined #archiveteam [09:50] *** underscor has joined #archiveteam [09:50] *** swebb sets mode: +o underscor [09:51] *** RealMarc has joined #archiveteam [09:51] *** Elegance has joined #archiveteam [09:51] *** torvik has joined #archiveteam [09:51] *** SadDM has joined #archiveteam [09:51] *** nico_32 has joined #archiveteam [09:51] *** will__ has joined #archiveteam [09:51] *** wacky has joined #archiveteam [09:51] *** maltris has joined #archiveteam [09:51] *** rduser has joined #archiveteam [09:51] *** everdred has joined #archiveteam [09:51] *** ryan__ has joined #archiveteam [09:51] *** matthusb- has joined #archiveteam [09:51] *** irc.shaw.ca sets mode: +o SadDM [09:51] *** swebb sets mode: +o SadDM [09:51] *** maltris has quit IRC (Remote host closed the connection) [09:52] *** DFJustin has joined #archiveteam [09:52] *** swebb sets mode: +o DFJustin [09:53] *** dashcloud has joined #archiveteam [09:55] *** RealMarc has quit IRC (Ping timeout: 382 seconds) [09:55] *** will__ has quit IRC (Ping timeout: 382 seconds) [10:00] *** Marc- has quit IRC (Ping timeout: 240 seconds) [10:01] *** khaoohs_ has joined #archiveteam [10:01] *** khaoohs__ has quit IRC (Read error: Connection reset by peer) [10:02] *** will__ has joined #archiveteam [10:02] *** offby1 has joined #archiveteam [10:04] *** trs80 has joined #archiveteam [10:08] *** Fusl has joined #archiveteam [10:08] *** Emcy_ has quit IRC (Ping timeout: 265 seconds) [10:09] *** Emcy has joined #archiveteam [10:15] *** pikhq has joined #archiveteam [10:20] *** norbert79 has left [10:26] *** RealMarc has joined #archiveteam [10:52] *** BlueMaxim has quit IRC (Quit: Leaving) [10:55] *** xk_id has joined #archiveteam [10:55] *** xk_id_ has quit IRC (Read error: Connection reset by peer) [10:59] *** dashcloud has quit IRC (Read error: Operation timed out) [11:05] *** dashcloud has joined #archiveteam [11:33] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [12:09] *** useretail has joined #archiveteam [12:50] https://tweakers.net/nieuws/100669/tripadvisor-neemt-nederlandse-restaurantsite-iens-over.html [12:51] Dutch restaurant review site acquired by tripadvisor [12:51] future unclear; may either continue to exist separately, or be 'merged' into tripadvisor (but probably with data loss) [12:58] *** Ymgve has joined #archiveteam [13:09] *** signius has quit IRC (Ping timeout: 258 seconds) [13:22] *** signius has joined #archiveteam [13:23] *** APerti has joined #archiveteam [13:27] *** ruukasu has joined #archiveteam [13:46] *** dashcloud has quit IRC (Read error: Operation timed out) [13:46] *** sankin has joined #archiveteam [13:52] *** dashcloud has joined #archiveteam [14:11] *** dashcloud has quit IRC (Read error: Connection reset by peer) [14:16] *** dashcloud has joined #archiveteam [14:26] *** APerti has quit IRC (Ping timeout: 370 seconds) [14:29] *** Spring has joined #archiveteam [14:31] *** nertzy has joined #archiveteam [14:31] *** Start has joined #archiveteam [14:33] *** Specular has quit IRC (Ping timeout: 365 seconds) [14:59] *** APerti has joined #archiveteam [15:06] *** ruukasu has quit IRC (Quit: WeeChat 1.0.1) [15:06] *** ruukasu has joined #archiveteam [15:09] *** Start has quit IRC (Ping timeout: 265 seconds) [15:15] *** ruukasu has quit IRC (Quit: WeeChat 1.0.1) [15:18] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [15:21] *** ruukasu has joined #archiveteam [15:24] *** Specular has joined #archiveteam [15:25] *** ruukasuu has joined #archiveteam [15:25] *** Spring has quit IRC (Read error: Connection reset by peer) [15:27] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [15:27] *** ruukasuu has quit IRC (Client Quit) [15:27] *** K4k has joined #archiveteam [15:29] *** ruukasu has joined #archiveteam [15:30] *** Spring has joined #archiveteam [15:31] *** Specular has quit IRC (Read error: Connection reset by peer) [15:33] *** mistym has joined #archiveteam [15:36] *** mistym has quit IRC (Remote host closed the connection) [15:39] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [15:43] *** ruukasu has joined #archiveteam [15:51] *** aaaaaaaaa has joined #archiveteam [15:54] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [15:57] *** mistym has joined #archiveteam [16:02] *** APerti has quit IRC (Ping timeout: 370 seconds) [16:03] *** Start has joined #archiveteam [16:05] *** ruukasu has joined #archiveteam [16:09] *** rejon has joined #archiveteam [16:10] *** APerti has joined #archiveteam [16:22] *** nertzy has joined #archiveteam [16:23] *** dashcloud has quit IRC (Read error: Operation timed out) [16:39] *** dashcloud has joined #archiveteam [16:42] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [16:43] *** ruukasu has quit IRC (Quit: WeeChat 1.0.1) [16:46] *** nertzy has joined #archiveteam [16:50] *** mistym has quit IRC (Remote host closed the connection) [16:50] *** __uu has quit IRC (Ping timeout: 265 seconds) [16:51] *** __uu has joined #archiveteam [16:54] *** primus104 has quit IRC (Leaving.) [16:56] *** Start has quit IRC (Ping timeout: 265 seconds) [16:58] *** Start has joined #archiveteam [17:08] *** mistym has joined #archiveteam [17:10] *** Spring has quit IRC (Read error: Connection reset by peer) [17:11] *** Spring has joined #archiveteam [17:13] *** ruukasu has joined #archiveteam [17:14] *** Specular has joined #archiveteam [17:14] *** Spring has quit IRC (Read error: Connection reset by peer) [17:14] *** xtr-201 has quit IRC (Read error: Operation timed out) [17:15] *** Spring has joined #archiveteam [17:20] *** Specular has quit IRC (Ping timeout: 369 seconds) [17:31] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [17:51] *** Start has quit IRC (Ping timeout: 365 seconds) [17:52] *** garyrh has quit IRC (Quit: http://bnc4free.com/) [18:01] *** garyrh has joined #archiveteam [18:06] *** garyrh has quit IRC (Read error: Operation timed out) [18:08] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [18:13] *** philpem has joined #archiveteam [18:14] *** garyrh has joined #archiveteam [18:22] *** ruukasu has joined #archiveteam [18:25] *** abartov has quit IRC (Ping timeout: 258 seconds) [18:28] *** aaaaaaaa_ has joined #archiveteam [18:30] *** garyrh has quit IRC (Read error: Operation timed out) [18:31] *** aaaaaaaa_ has quit IRC (Client Quit) [18:31] *** arb735 has joined #archiveteam [18:32] *** arb735 has quit IRC (Client Quit) [18:33] *** aaaaaaaaa has quit IRC (Read error: Operation timed out) [18:41] *** primus104 has joined #archiveteam [18:44] *** abartov has joined #archiveteam [18:49] *** Start has joined #archiveteam [19:02] *** garyrh has joined #archiveteam [19:04] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [19:08] *** ruukasu has joined #archiveteam [19:11] *** rejon has quit IRC (Read error: Operation timed out) [19:28] *** ionpulse has quit IRC (Ping timeout: 512 seconds) [19:37] *** K4k has quit IRC (Quit: WeeChat 1.0.1) [19:38] *** aaaaaaaaa has joined #archiveteam [19:38] *** K4k has joined #archiveteam [19:43] *** Start has quit IRC (Ping timeout: 365 seconds) [19:49] *** mistym has quit IRC (Remote host closed the connection) [20:03] *** Specular has joined #archiveteam [20:03] *** Spring has quit IRC (Read error: Connection reset by peer) [20:06] *** ruukasu has quit IRC (Ping timeout: 265 seconds) [20:11] *** BlueMaxim has joined #archiveteam [20:17] *** APerti has quit IRC (Ping timeout: 370 seconds) [20:27] *** ruukasu has joined #archiveteam [20:31] *** Start has joined #archiveteam [20:44] *** nertzy has joined #archiveteam [20:44] *** rejon has joined #archiveteam [20:51] *** lytv has quit IRC (Read error: Operation timed out) [20:55] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [20:56] *** mistym has joined #archiveteam [20:59] *** nertzy has joined #archiveteam [21:07] *** APerti has joined #archiveteam [21:17] *** lytv has joined #archiveteam [21:21] *** Start has quit IRC (Read error: Connection reset by peer) [21:21] *** ruukasu has quit IRC (Quit: WeeChat 1.0.1) [21:26] *** Specular has quit IRC (Ping timeout: 365 seconds) [21:29] *** bauruine has quit IRC (Ping timeout: 265 seconds) [21:31] *** nertzy has quit IRC (Quit: This computer has gone to sleep) [21:56] *** sankin has quit IRC (Leaving.) [22:14] *** ruukasu has joined #archiveteam [22:30] *** K4k has quit IRC (Read error: Operation timed out) [22:31] *** mistym has quit IRC (Remote host closed the connection) [22:40] We're dropping this on the world: [22:40] https://archive.org/details/audio-covers [23:17] *** bauruine has joined #archiveteam [23:33] *** abartov has quit IRC (Ping timeout: 258 seconds) [23:35] *** dashcloud has quit IRC (No Ping reply in 210 seconds.) [23:46] *** dashcloud has joined #archiveteam [23:46] *** kyan has quit IRC (Remote host closed the connection) [23:51] *** kyan has joined #archiveteam