| Time |
Nickname |
Message |
|
00:59
🔗
|
omf_ |
Just to make sure I am covering my bases |
|
00:59
🔗
|
omf_ |
What other scrapers are people familar with or have used |
|
00:59
🔗
|
omf_ |
besides curl and aria2c |
|
10:02
🔗
|
omf_ |
has anyone else tried |
|
10:03
🔗
|
omf_ |
wget -c -m --warc-file --warc-cdx |
|
10:03
🔗
|
omf_ |
I just tried it out and it erased the old warc and started over |
|
10:04
🔗
|
ersi |
If you --warc-file $name an existing file, it'll over write it |
|
10:11
🔗
|
omf_ |
so would creating a new file name allow it to continue where it left off? |
|
10:12
🔗
|
ersi |
I guess not :) |
|
10:12
🔗
|
omf_ |
couldn't we just join the warcs later |
|
10:13
🔗
|
ersi |
yah, that's no prob |
|
10:13
🔗
|
omf_ |
well let me try this out |
|
10:14
🔗
|
omf_ |
I am testing the limits of what I can do with wget |