#archiveteam 2018-01-31,Wed

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)

WhoWhatWhen
***Aranje has quit IRC (Read error: Operation timed out)
Aranje has joined #archiveteam
[00:01]
.......... (idle for 49mn)
jacketcha has joined #archiveteam [00:51]
Pixi has quit IRC (Quit: Pixi) [01:01]
Pixi has joined #archiveteam [01:08]
..... (idle for 20mn)
pizzaiolo has quit IRC (Ping timeout: 246 seconds)
pizzaiolo has joined #archiveteam
Selanda_ has quit IRC (Ping timeout: 260 seconds)
Selanda has joined #archiveteam
pizzaiolo has quit IRC (Client Quit)
pizzaiolo has joined #archiveteam
[01:28]
pizzaiolo has quit IRC (Remote host closed the connection)
BlueMaxim has quit IRC (Leaving)
ranavalon has quit IRC (Read error: Connection reset by peer)
ranavalon has joined #archiveteam
ranavalon has quit IRC (Remote host closed the connection)
ranavalon has joined #archiveteam
[01:42]
kitties has joined #archiveteam
jacketcha has quit IRC (Read error: Connection reset by peer)
[01:59]
nertzy has quit IRC (Read error: Connection reset by peer)
jacketcha has joined #archiveteam
[02:08]
BlueMaxim has joined #archiveteam [02:22]
mistym has joined #archiveteam [02:31]
....... (idle for 34mn)
Aranje has quit IRC (Read error: Operation timed out)
Aranje has joined #archiveteam
[03:05]
Aranje has quit IRC (Read error: Operation timed out)
dx has joined #archiveteam
Aranje has joined #archiveteam
parker has joined #archiveteam
dx has left
[03:10]
parker has quit IRC (Read error: Operation timed out)
BlueMaxim has quit IRC (Leaving)
parker has joined #archiveteam
[03:29]
.... (idle for 17mn)
BlueMaxim has joined #archiveteam [03:51]
...... (idle for 27mn)
parker has quit IRC (Ping timeout: 360 seconds) [04:18]
...... (idle for 27mn)
qw3rty118 has joined #archiveteam
jacketcha has quit IRC (Read error: Connection reset by peer)
jacketcha has joined #archiveteam
nwf__ has quit IRC (WeeChat 1.6)
qw3rty117 has quit IRC (Read error: Operation timed out)
Aranje has quit IRC (Quit: Three sheets to the wind)
[04:45]
.... (idle for 17mn)
ranav has joined #archiveteam
zhongfu has quit IRC (Remote host closed the connection)
mona has quit IRC (Ping timeout: 260 seconds)
ranavalon has quit IRC (Read error: Operation timed out)
mona has joined #archiveteam
[05:08]
Ctrl has quit IRC (Ping timeout: 506 seconds)
nwf has joined #archiveteam
[05:22]
..... (idle for 23mn)
antomatic has quit IRC (Ping timeout: 252 seconds)
jacketcha has quit IRC (Read error: Connection reset by peer)
jacketcha has joined #archiveteam
[05:47]
.... (idle for 18mn)
Stilett0 is now known as Stiletto [06:06]
..... (idle for 24mn)
antomatic has joined #archiveteam [06:30]
antomatic has quit IRC (Read error: Operation timed out)
antomatic has joined #archiveteam
[06:40]
.......... (idle for 45mn)
pikhq has quit IRC (Ping timeout: 250 seconds)
BobJonkma has joined #archiveteam
[07:25]
pikhq has joined #archiveteam [07:40]
kitties has quit IRC (Quit: Connection closed for inactivity) [07:45]
..................... (idle for 1h44mn)
atomotic has joined #archiveteam [09:29]
......... (idle for 44mn)
schbirid has joined #archiveteam
SilSte has quit IRC (Read error: Connection reset by peer)
[10:13]
.... (idle for 17mn)
Mateon1 has quit IRC (Read error: Operation timed out)
Mateon1 has joined #archiveteam
[10:32]
...................... (idle for 1h48mn)
BlueMaxim has quit IRC (Leaving) [12:21]
........ (idle for 37mn)
atomotic has quit IRC (Quit: atomotic) [12:58]
.......... (idle for 47mn)
atomotic has joined #archiveteam [13:45]
Morbus has joined #archiveteam [13:55]
........ (idle for 37mn)
atomotic has quit IRC (Quit: atomotic) [14:32]
................. (idle for 1h20mn)
ld1 has quit IRC (Quit: ld1)
ld1 has joined #archiveteam
[15:52]
.... (idle for 19mn)
atomotic has joined #archiveteam [16:14]
parker has joined #archiveteam
RichardG has quit IRC (Ping timeout: 506 seconds)
[16:26]
parker has quit IRC (Read error: Operation timed out) [16:42]
parker has joined #archiveteam [16:48]
atomotic has quit IRC (Quit: atomotic) [17:01]
atrocity has quit IRC () [17:10]
RichardG has joined #archiveteam [17:15]
atomotic has joined #archiveteam [17:23]
...... (idle for 28mn)
atomotic has quit IRC (Quit: atomotic) [17:51]
..... (idle for 21mn)
jschwart has joined #archiveteam [18:12]
.............. (idle for 1h6mn)
parker has quit IRC (Ping timeout: 360 seconds) [19:18]
...... (idle for 28mn)
octothorp has quit IRC (Read error: Connection reset by peer)
octothorp has joined #archiveteam
[19:46]
Soni has quit IRC (Ping timeout: 255 seconds) [19:57]
Morbus has quit IRC (Ping timeout: 255 seconds)
Morbus has joined #archiveteam
[20:09]
Soni has joined #archiveteam [20:16]
.... (idle for 15mn)
parker has joined #archiveteam
Scippy has joined #archiveteam
Scippy has quit IRC (Ping timeout: 260 seconds)
RichardG has quit IRC (Read error: Connection reset by peer)
[20:31]
n00b604 has joined #archiveteam [20:45]
n00b604Hi. Is there any tool for crawling and creating a WARC of a website that requires JS? I think already have a way of enumerating all the pages. [20:47]
WubTheCapArchiveBot supports PhantomJS, if you mean infinite scrolling [20:48]
n00b604WubTheCap: can it save a page like this <https://arbital.com/p/bayes_rule/>?
Wayback can't do it: http://web.archive.org/web/20171101121322/https://arbital.com/p/bayes_rule/
Even archive.is can't: http://archive.is/6Kxrl
gotta go, might be back later (and will read the log to see if anyone has ideas)
btw the reason I want to archive this is because it apparently is being shut down. they say they'll maintain an archive, but I want to make sure
[20:49]
***n00b604 has quit IRC (Quit: Page closed) [20:52]
.... (idle for 15mn)
RichardG has joined #archiveteam [21:07]
.... (idle for 16mn)
JAAArchiveBot *should* support PhantomJS for scrolling (e.g. Twitter), but that has been broken on most pipelines for many months.
You can use a browser with a WARC-writing MITM proxy such as warcprox. That would capture exactly what the browser requests and receives, including anything loaded after the initial page load etc.
But that's entirely manual. I don't have experience with working automatic tools; I know that brozzler exists though, which is warcprox + Chromium headless, I believe, so you could give that a try.
(NB, I didn't look at that page at all.)
[21:23]
***don is now known as don_
Atom-- has joined #archiveteam
[21:31]
WubTheCap has quit IRC (Read error: Connection reset by peer)
Atom has quit IRC (Read error: Operation timed out)
WubTheCap has joined #archiveteam
Atom-- has quit IRC (Read error: Operation timed out)
[21:40]
.... (idle for 15mn)
pizzaiolo has joined #archiveteam [22:01]
pizzaiolo has quit IRC (Read error: Connection reset by peer)
pizzaiolo has joined #archiveteam
[22:15]
schbirid has quit IRC (Quit: Leaving) [22:22]
.... (idle for 16mn)
jschwart has quit IRC (Quit: Konversation terminated!)
machina has joined #archiveteam
[22:38]
pizzaiolo has quit IRC (Remote host closed the connection) [22:46]
soldaat has joined #archiveteam
soldaat has quit IRC (Client Quit)
[22:52]
...... (idle for 26mn)
MrDignity has joined #archiveteam [23:20]
BlueMaxim has joined #archiveteam [23:28]

↑back Search ←Prev date Next date→ Show only urls(Click on time to select a line by its url)