| Time |
Nickname |
Message |
|
00:06
🔗
|
|
betamax_o has quit IRC (leaving) |
|
00:23
🔗
|
|
Dragnog2 has quit IRC (Quit: Connection closed for inactivity) |
|
01:23
🔗
|
|
Pixi has quit IRC (Quit: Pixi) |
|
01:24
🔗
|
|
Pixi has joined #archiveteam-bs |
|
01:26
🔗
|
|
d5f4a3622 has quit IRC (https://i.imgur.com/xacQ09F.mp4) |
|
01:28
🔗
|
|
d5f4a3622 has joined #archiveteam-bs |
|
01:52
🔗
|
|
Gfy has quit IRC (Ping timeout: 258 seconds) |
|
01:53
🔗
|
|
tuluu has quit IRC (Ping timeout: 258 seconds) |
|
01:55
🔗
|
|
Lord_Nigh has quit IRC (Ping timeout: 258 seconds) |
|
01:55
🔗
|
|
Lord_Nigh has joined #archiveteam-bs |
|
01:55
🔗
|
|
mls has quit IRC (Ping timeout: 258 seconds) |
|
01:57
🔗
|
|
anarcat has joined #archiveteam-bs |
|
01:57
🔗
|
|
anarcat has quit IRC (Handshake flooding) |
|
01:58
🔗
|
|
anarchat has quit IRC (Ping timeout: 258 seconds) |
|
01:58
🔗
|
|
tuluu has joined #archiveteam-bs |
|
02:01
🔗
|
|
d5f4a3622 has quit IRC (Ping timeout: 258 seconds) |
|
02:02
🔗
|
|
d5f4a3622 has joined #archiveteam-bs |
|
02:02
🔗
|
|
anarcat has joined #archiveteam-bs |
|
02:02
🔗
|
|
anarcat has quit IRC (Handshake flooding) |
|
02:07
🔗
|
|
anarcat has joined #archiveteam-bs |
|
02:10
🔗
|
|
omglolbah has quit IRC (se.hub efnet.portlane.se) |
|
02:10
🔗
|
|
Laverne_ has quit IRC (se.hub efnet.portlane.se) |
|
02:10
🔗
|
|
Shen has quit IRC (se.hub efnet.portlane.se) |
|
02:17
🔗
|
|
manjaro-u has quit IRC (Read error: Operation timed out) |
|
02:48
🔗
|
|
Gfy has joined #archiveteam-bs |
|
02:48
🔗
|
|
mls has joined #archiveteam-bs |
|
02:48
🔗
|
|
omglolbah has joined #archiveteam-bs |
|
02:48
🔗
|
|
Laverne_ has joined #archiveteam-bs |
|
02:48
🔗
|
|
Shen has joined #archiveteam-bs |
|
04:37
🔗
|
|
X-Scale` has joined #archiveteam-bs |
|
04:38
🔗
|
|
odemgi has joined #archiveteam-bs |
|
04:39
🔗
|
|
X-Scale has quit IRC (Ping timeout: 252 seconds) |
|
04:39
🔗
|
|
X-Scale` is now known as X-Scale |
|
04:40
🔗
|
|
odemgi_ has quit IRC (Read error: Operation timed out) |
|
04:46
🔗
|
|
qw3rty2 has joined #archiveteam-bs |
|
04:49
🔗
|
|
odemg has quit IRC (Ping timeout: 745 seconds) |
|
04:52
🔗
|
|
IAmbience has quit IRC (Quit: Connection closed for inactivity) |
|
04:53
🔗
|
|
X-Scale` has joined #archiveteam-bs |
|
04:54
🔗
|
|
odemg has joined #archiveteam-bs |
|
04:54
🔗
|
|
qw3rty has quit IRC (Ping timeout: 745 seconds) |
|
04:54
🔗
|
|
X-Scale has quit IRC (Ping timeout: 252 seconds) |
|
04:54
🔗
|
|
X-Scale` is now known as X-Scale |
|
05:10
🔗
|
|
ugh has joined #archiveteam-bs |
|
05:13
🔗
|
|
legoktm has quit IRC (Read error: Operation timed out) |
|
05:44
🔗
|
|
asdf0101 has quit IRC (The Lounge - https://thelounge.chat) |
|
05:44
🔗
|
|
markedL has quit IRC (Quit: The Lounge - https://thelounge.chat) |
|
05:45
🔗
|
|
markedL has joined #archiveteam-bs |
|
05:45
🔗
|
|
asdf0101 has joined #archiveteam-bs |
|
07:40
🔗
|
|
BlueMax has quit IRC (Quit: Leaving) |
|
08:27
🔗
|
|
d5f4a3622 has quit IRC (Quit: https://i.imgur.com/xacQ09F.mp4) |
|
08:30
🔗
|
|
d5f4a3622 has joined #archiveteam-bs |
|
08:57
🔗
|
|
kiska18 has quit IRC (Remote host closed the connection) |
|
08:57
🔗
|
|
Ryz has quit IRC (Remote host closed the connection) |
|
08:57
🔗
|
|
kiska18 has joined #archiveteam-bs |
|
08:57
🔗
|
|
Fusl sets mode: +o kiska18 |
|
08:57
🔗
|
|
Fusl__ sets mode: +o kiska18 |
|
08:57
🔗
|
|
Fusl_ sets mode: +o kiska18 |
|
08:57
🔗
|
|
Ryz has joined #archiveteam-bs |
|
08:58
🔗
|
|
schbirid has joined #archiveteam-bs |
|
08:59
🔗
|
|
Ing3b0rg has quit IRC (Quit: woopwoop) |
|
09:00
🔗
|
|
Ing3b0rg has joined #archiveteam-bs |
|
09:49
🔗
|
|
britm0b has quit IRC (Read error: Connection reset by peer) |
|
09:50
🔗
|
|
britm0b has joined #archiveteam-bs |
|
10:50
🔗
|
|
IAmbience has joined #archiveteam-bs |
|
11:41
🔗
|
|
Piep has joined #archiveteam-bs |
|
11:41
🔗
|
|
Piep has quit IRC (Client Quit) |
|
12:00
🔗
|
|
Damme_ is now known as Damme |
|
13:13
🔗
|
|
tuluu has quit IRC (Remote host closed the connection) |
|
13:14
🔗
|
|
tuluu has joined #archiveteam-bs |
|
13:25
🔗
|
|
vitzli has joined #archiveteam-bs |
|
14:02
🔗
|
|
Smiley has quit IRC (Read error: Connection reset by peer) |
|
14:02
🔗
|
|
Smiley has joined #archiveteam-bs |
|
14:08
🔗
|
|
SmileyG has joined #archiveteam-bs |
|
14:10
🔗
|
|
Smiley has quit IRC (Ping timeout: 255 seconds) |
|
15:01
🔗
|
|
vitzli has quit IRC (Quit: Leaving) |
|
15:14
🔗
|
britm0b |
I am using grab-site to grab a website. It says on the web ui that it has grabbed 13920MB, but the WARC file is only 2.6G and there are no WARCs in the finished WARC dir. Is this due to dupes? |
|
15:20
🔗
|
JAA |
britm0b: No, compression. The 13920 MB (really MiB I believe) is how much has been downloaded, and that apparently compressed down to 2.6 GiB of WARCs. Since the max WARC size is larger, nothing was moved to the finished dir yet. |
|
15:24
🔗
|
britm0b |
Ah that makes sense. I forgot how much text can be compressed when it's mostly the same! :) |
|
15:27
🔗
|
|
bluefoo has quit IRC (Remote host closed the connection) |
|
15:30
🔗
|
|
bluefoo has joined #archiveteam-bs |
|
15:52
🔗
|
|
killsushi has quit IRC (Quit: Leaving) |
|
16:07
🔗
|
markedL |
someone correct me if I'm wrong, each record is compressed individually, so the similarity between pages is not part of the efficiency? |
|
17:07
🔗
|
|
Hani has joined #archiveteam-bs |
|
17:26
🔗
|
JAA |
Correct. |
|
17:26
🔗
|
JAA |
And also the similarity between the WARC headers etc. |
|
17:26
🔗
|
JAA |
Otherwise, the file would be much smaller still. |
|
17:27
🔗
|
|
d5f4a3622 has quit IRC (Quit: https://i.imgur.com/xacQ09F.mp4) |
|
17:27
🔗
|
JAA |
But HTML/JS/CSS is still highly compressible of course. |
|
17:32
🔗
|
|
d5f4a3622 has joined #archiveteam-bs |
|
17:37
🔗
|
|
d5f4a3622 has quit IRC (Quit: https://i.imgur.com/xacQ09F.mp4) |
|
17:42
🔗
|
|
DogsRNice has joined #archiveteam-bs |
|
17:48
🔗
|
|
icedice has joined #archiveteam-bs |
|
18:25
🔗
|
|
d5f4a3622 has joined #archiveteam-bs |
|
18:26
🔗
|
|
kiska18 has quit IRC (Read error: Operation timed out) |
|
18:27
🔗
|
|
Ryz has quit IRC (Quit: Ping timeout (120 seconds)) |
|
18:28
🔗
|
|
Ryz has joined #archiveteam-bs |
|
18:33
🔗
|
|
Ryz has quit IRC (Quit: Ping timeout (120 seconds)) |
|
18:33
🔗
|
|
Ryz has joined #archiveteam-bs |
|
18:56
🔗
|
|
Ivy has quit IRC (Quit: Connection closed for inactivity) |
|
19:32
🔗
|
|
tomaspark has joined #archiveteam-bs |
|
21:46
🔗
|
|
manjaro-u has joined #archiveteam-bs |
|
21:49
🔗
|
|
icedice has quit IRC (Quit: Leaving) |
|
21:51
🔗
|
|
schbirid has quit IRC (Quit: Leaving) |
|
22:27
🔗
|
|
BlueMax has joined #archiveteam-bs |
|
23:54
🔗
|
|
phirephly has quit IRC (Read error: Operation timed out) |