#archiveteam 2018-01-31,Wed

↑back Search

Time Nickname Message
00:01 🔗 Aranje has quit IRC (Read error: Operation timed out)
00:02 🔗 Aranje has joined #archiveteam
00:51 🔗 jacketcha has joined #archiveteam
01:01 🔗 Pixi has quit IRC (Quit: Pixi)
01:08 🔗 Pixi has joined #archiveteam
01:28 🔗 pizzaiolo has quit IRC (Ping timeout: 246 seconds)
01:29 🔗 pizzaiolo has joined #archiveteam
01:30 🔗 Selanda_ has quit IRC (Ping timeout: 260 seconds)
01:31 🔗 Selanda has joined #archiveteam
01:33 🔗 pizzaiolo has quit IRC (Client Quit)
01:33 🔗 pizzaiolo has joined #archiveteam
01:42 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
01:43 🔗 BlueMaxim has quit IRC (Leaving)
01:44 🔗 ranavalon has quit IRC (Read error: Connection reset by peer)
01:46 🔗 ranavalon has joined #archiveteam
01:47 🔗 ranavalon has quit IRC (Remote host closed the connection)
01:47 🔗 ranavalon has joined #archiveteam
01:59 🔗 kitties has joined #archiveteam
02:02 🔗 jacketcha has quit IRC (Read error: Connection reset by peer)
02:08 🔗 nertzy has quit IRC (Read error: Connection reset by peer)
02:11 🔗 jacketcha has joined #archiveteam
02:22 🔗 BlueMaxim has joined #archiveteam
02:31 🔗 mistym has joined #archiveteam
03:05 🔗 Aranje has quit IRC (Read error: Operation timed out)
03:05 🔗 Aranje has joined #archiveteam
03:10 🔗 Aranje has quit IRC (Read error: Operation timed out)
03:10 🔗 dx has joined #archiveteam
03:10 🔗 Aranje has joined #archiveteam
03:13 🔗 parker has joined #archiveteam
03:15 🔗 dx has left
03:29 🔗 parker has quit IRC (Read error: Operation timed out)
03:32 🔗 BlueMaxim has quit IRC (Leaving)
03:34 🔗 parker has joined #archiveteam
03:51 🔗 BlueMaxim has joined #archiveteam
04:18 🔗 parker has quit IRC (Ping timeout: 360 seconds)
04:45 🔗 qw3rty118 has joined #archiveteam
04:47 🔗 jacketcha has quit IRC (Read error: Connection reset by peer)
04:47 🔗 jacketcha has joined #archiveteam
04:49 🔗 nwf__ has quit IRC (WeeChat 1.6)
04:49 🔗 qw3rty117 has quit IRC (Read error: Operation timed out)
04:51 🔗 Aranje has quit IRC (Quit: Three sheets to the wind)
05:08 🔗 ranav has joined #archiveteam
05:12 🔗 zhongfu has quit IRC (Remote host closed the connection)
05:12 🔗 mona has quit IRC (Ping timeout: 260 seconds)
05:15 🔗 ranavalon has quit IRC (Read error: Operation timed out)
05:16 🔗 mona has joined #archiveteam
05:22 🔗 Ctrl has quit IRC (Ping timeout: 506 seconds)
05:24 🔗 nwf has joined #archiveteam
05:47 🔗 antomatic has quit IRC (Ping timeout: 252 seconds)
05:47 🔗 jacketcha has quit IRC (Read error: Connection reset by peer)
05:48 🔗 jacketcha has joined #archiveteam
06:06 🔗 Stilett0 is now known as Stiletto
06:30 🔗 antomatic has joined #archiveteam
06:40 🔗 antomatic has quit IRC (Read error: Operation timed out)
06:40 🔗 antomatic has joined #archiveteam
07:25 🔗 pikhq has quit IRC (Ping timeout: 250 seconds)
07:26 🔗 BobJonkma has joined #archiveteam
07:40 🔗 pikhq has joined #archiveteam
07:45 🔗 kitties has quit IRC (Quit: Connection closed for inactivity)
09:29 🔗 atomotic has joined #archiveteam
10:13 🔗 schbirid has joined #archiveteam
10:15 🔗 SilSte has quit IRC (Read error: Connection reset by peer)
10:32 🔗 Mateon1 has quit IRC (Read error: Operation timed out)
10:33 🔗 Mateon1 has joined #archiveteam
12:21 🔗 BlueMaxim has quit IRC (Leaving)
12:58 🔗 atomotic has quit IRC (Quit: atomotic)
13:45 🔗 atomotic has joined #archiveteam
13:55 🔗 Morbus has joined #archiveteam
14:32 🔗 atomotic has quit IRC (Quit: atomotic)
15:52 🔗 ld1 has quit IRC (Quit: ld1)
15:55 🔗 ld1 has joined #archiveteam
16:14 🔗 atomotic has joined #archiveteam
16:26 🔗 parker has joined #archiveteam
16:29 🔗 RichardG has quit IRC (Ping timeout: 506 seconds)
16:42 🔗 parker has quit IRC (Read error: Operation timed out)
16:48 🔗 parker has joined #archiveteam
17:01 🔗 atomotic has quit IRC (Quit: atomotic)
17:10 🔗 atrocity has quit IRC ()
17:15 🔗 RichardG has joined #archiveteam
17:23 🔗 atomotic has joined #archiveteam
17:51 🔗 atomotic has quit IRC (Quit: atomotic)
18:12 🔗 jschwart has joined #archiveteam
19:18 🔗 parker has quit IRC (Ping timeout: 360 seconds)
19:46 🔗 octothorp has quit IRC (Read error: Connection reset by peer)
19:47 🔗 octothorp has joined #archiveteam
19:57 🔗 Soni has quit IRC (Ping timeout: 255 seconds)
20:09 🔗 Morbus has quit IRC (Ping timeout: 255 seconds)
20:09 🔗 Morbus has joined #archiveteam
20:16 🔗 Soni has joined #archiveteam
20:31 🔗 parker has joined #archiveteam
20:33 🔗 Scippy has joined #archiveteam
20:37 🔗 Scippy has quit IRC (Ping timeout: 260 seconds)
20:38 🔗 RichardG has quit IRC (Read error: Connection reset by peer)
20:45 🔗 n00b604 has joined #archiveteam
20:47 🔗 n00b604 Hi. Is there any tool for crawling and creating a WARC of a website that requires JS? I think already have a way of enumerating all the pages.
20:48 🔗 WubTheCap ArchiveBot supports PhantomJS, if you mean infinite scrolling
20:49 🔗 n00b604 WubTheCap: can it save a page like this <https://arbital.com/p/bayes_rule/>?
20:50 🔗 n00b604 Wayback can't do it: http://web.archive.org/web/20171101121322/https://arbital.com/p/bayes_rule/
20:50 🔗 n00b604 Even archive.is can't: http://archive.is/6Kxrl
20:51 🔗 n00b604 gotta go, might be back later (and will read the log to see if anyone has ideas)
20:52 🔗 n00b604 btw the reason I want to archive this is because it apparently is being shut down. they say they'll maintain an archive, but I want to make sure
20:52 🔗 n00b604 has quit IRC (Quit: Page closed)
21:07 🔗 RichardG has joined #archiveteam
21:23 🔗 JAA ArchiveBot *should* support PhantomJS for scrolling (e.g. Twitter), but that has been broken on most pipelines for many months.
21:23 🔗 JAA You can use a browser with a WARC-writing MITM proxy such as warcprox. That would capture exactly what the browser requests and receives, including anything loaded after the initial page load etc.
21:25 🔗 JAA But that's entirely manual. I don't have experience with working automatic tools; I know that brozzler exists though, which is warcprox + Chromium headless, I believe, so you could give that a try.
21:25 🔗 JAA (NB, I didn't look at that page at all.)
21:31 🔗 don is now known as don_
21:35 🔗 Atom-- has joined #archiveteam
21:40 🔗 WubTheCap has quit IRC (Read error: Connection reset by peer)
21:41 🔗 Atom has quit IRC (Read error: Operation timed out)
21:44 🔗 WubTheCap has joined #archiveteam
21:46 🔗 Atom-- has quit IRC (Read error: Operation timed out)
22:01 🔗 pizzaiolo has joined #archiveteam
22:15 🔗 pizzaiolo has quit IRC (Read error: Connection reset by peer)
22:16 🔗 pizzaiolo has joined #archiveteam
22:22 🔗 schbirid has quit IRC (Quit: Leaving)
22:38 🔗 jschwart has quit IRC (Quit: Konversation terminated!)
22:38 🔗 machina has joined #archiveteam
22:46 🔗 pizzaiolo has quit IRC (Remote host closed the connection)
22:52 🔗 soldaat has joined #archiveteam
22:54 🔗 soldaat has quit IRC (Client Quit)
23:20 🔗 MrDignity has joined #archiveteam
23:28 🔗 BlueMaxim has joined #archiveteam

irclogger-viewer