| Time |
Nickname |
Message |
|
00:30
🔗
|
|
Soni has quit IRC (Remote host closed the connection) |
|
00:38
🔗
|
|
Soni has joined #archiveteam-bs |
|
00:42
🔗
|
|
BlueMax has joined #archiveteam-bs |
|
02:44
🔗
|
dashcloud |
I'm thinking it might be several shows? |
|
03:03
🔗
|
|
Valentine has quit IRC (Ping timeout: 506 seconds) |
|
03:42
🔗
|
|
qw3rty114 has joined #archiveteam-bs |
|
03:47
🔗
|
|
qw3rty113 has quit IRC (Read error: Operation timed out) |
|
03:49
🔗
|
|
odemg has quit IRC (Read error: Operation timed out) |
|
03:54
🔗
|
|
Mateon1 has quit IRC (Read error: Operation timed out) |
|
03:54
🔗
|
|
Mateon1 has joined #archiveteam-bs |
|
03:55
🔗
|
|
ndiddy has quit IRC () |
|
03:59
🔗
|
|
odemg has joined #archiveteam-bs |
|
10:57
🔗
|
|
zhongfu has joined #archiveteam-bs |
|
11:06
🔗
|
|
BlueMax has quit IRC (Leaving) |
|
11:53
🔗
|
|
Sk1d has joined #archiveteam-bs |
|
14:28
🔗
|
|
MrRadar2 has quit IRC (Read error: Operation timed out) |
|
14:30
🔗
|
|
yuitimoth has quit IRC (Ping timeout: 260 seconds) |
|
15:02
🔗
|
|
MrRadar2 has joined #archiveteam-bs |
|
15:03
🔗
|
|
svchfoo3 sets mode: +o MrRadar2 |
|
15:43
🔗
|
|
ola_norsk has joined #archiveteam-bs |
|
15:44
🔗
|
ola_norsk |
isn't this "force a quick crawl" exactly what #archivebot does? :D https://archive.org/post/1090545/fee-based-enhanced-services |
|
15:45
🔗
|
ola_norsk |
(not that it might not be a cool way to get businesses to open their wallet..) |
|
16:11
🔗
|
ivan |
ola_norsk: IA runs Archive-It |
|
16:15
🔗
|
ola_norsk |
oh. well there's his or her's answer then |
|
16:25
🔗
|
ola_norsk |
https://www.irs.gov/taxonomy/term/17426 |
|
16:31
🔗
|
ola_norsk |
dangit |
|
16:31
🔗
|
ola_norsk |
sorry, drag-n-dropped that to wrong window somehow |
|
16:33
🔗
|
|
ola_norsk has quit IRC (leaving) |
|
16:44
🔗
|
|
wp494 has quit IRC (Ping timeout: 260 seconds) |
|
16:46
🔗
|
|
wp494 has joined #archiveteam-bs |
|
16:47
🔗
|
|
svchfoo1 sets mode: +o wp494 |
|
16:48
🔗
|
|
schbirid has joined #archiveteam-bs |
|
17:01
🔗
|
JAA |
"134 GB" of footage from the Syrian Civil War: https://old.reddit.com/r/CombatFootage/comments/8jcgb3/i_have_made_part_of_my_syrian_civil_war_archive/ |
|
17:42
🔗
|
godane |
SketchCow: any news about the tapes? |
|
18:22
🔗
|
|
Valentine has joined #archiveteam-bs |
|
18:33
🔗
|
|
Valentine has quit IRC (Quit: Addio, adieu, adios, aloha, arrivederci, auf Wiedersehen, au revoir, bye, bye-bye, cheerio, cheers, farewell, good) |
|
18:37
🔗
|
|
Valentine has joined #archiveteam-bs |
|
18:49
🔗
|
|
tyzoid has joined #archiveteam-bs |
|
18:49
🔗
|
tyzoid |
sounds good |
|
18:50
🔗
|
tyzoid |
(conv with JAA continuing from #archiveteam) |
|
18:51
🔗
|
tyzoid |
So do you think pipeline might be a better option, in this case? |
|
18:55
🔗
|
JAA |
Yeah, whenever you want to run multiple warriors in parallel, using the scripts themselves is probably a better idea in my opinion. |
|
18:56
🔗
|
JAA |
Also if you want to support multiple projects at once, rather than only the default one. |
|
18:57
🔗
|
JAA |
However, the downside of running scripts directly is that you have to do everything manually. |
|
18:57
🔗
|
JAA |
Joining new projects, updating the code, etc. |
|
18:58
🔗
|
JAA |
A possible alternative for you might be to run a VM (with Debian, Ubuntu, whichever you prefer) and then the Docker warrior multiple times inside that VM. |
|
19:02
🔗
|
tyzoid |
that's possible too, though I feel bad about running containers in vms when I've got a container manager already xD |
|
19:02
🔗
|
tyzoid |
so JAA: I'm thinking of setting up a pipeline instead, then it'll be able to process whatever. |
|
19:03
🔗
|
JAA |
"Pipeline"'s a very general concept used inside the warrior but also on ArchiveBot. |
|
19:04
🔗
|
tyzoid |
Ah, I'm referring to Archivebot pipeline |
|
19:04
🔗
|
tyzoid |
"As of November 2017, ArchiveBot has again started accepting applications from volunteers who want to set up new pipelines" |
|
19:05
🔗
|
JAA |
Right, but that has nothing at all to do with the warrior. |
|
19:06
🔗
|
tyzoid |
My goal wasn't to run warrior. I've done that in the past, though. My goal is to set up something that'll be helpful, but won't require a lot of manual intervention |
|
19:06
🔗
|
tyzoid |
multiple ip addresses is a bonus, if supported |
|
19:07
🔗
|
tyzoid |
(I've got an entire /48 that I route portions of, and two free ipv4 addresses) |
|
19:08
🔗
|
JAA |
ArchiveBot pipelines aren't exactly low-maintenance, unfortunately. Not as low maintenance as warrior VMs, at least. |
|
19:09
🔗
|
tyzoid |
Hmm |
|
19:09
🔗
|
JAA |
Also, IPv6 isn't really supported anywhere yet as far as I know. ArchiveBot definitely doesn't use it. |
|
19:09
🔗
|
tyzoid |
I see. |
|
19:09
🔗
|
tyzoid |
I could give root access to the container, since it'd be like a vps |
|
19:10
🔗
|
tyzoid |
but it might just be that I'd need to run multiple warriors to take advantage of ips |
|
19:13
🔗
|
tyzoid |
Is that a deliberate choice to not support ipv6? or is that just a thing that hasn't really been taken advantage of yet? |
|
19:19
🔗
|
JAA |
The latter, I think. |
|
19:20
🔗
|
JAA |
For ArchiveBot, I recently asked about what's needed for IPv6 support because I want to add it: https://github.com/ArchiveTeam/ArchiveBot/issues/315 |
|
19:23
🔗
|
tyzoid |
JAA: I assume it's because some sites might return different results on ipv4/ipv6, esp. if ipv6 is temporarily broken. |
|
19:23
🔗
|
tyzoid |
I know it's happened on my sites. I try to maintain ipv6, but it sometimes breaks. |
|
19:24
🔗
|
tyzoid |
shouldn't be too big an issue for larger sites, esp. if a positive result is returned |
|
19:24
🔗
|
tyzoid |
but that's just my guess |
|
19:30
🔗
|
tyzoid |
JAA: is there support for having multiple admins on a pipeline server? Like if something breaks, could I give key access to one of the team leads here to fix it? |
|
19:38
🔗
|
JAA |
tyzoid: Yeah, that is an issue. However, more and more end users have IPv6 support nowadays, so arguably, that's what we should grab. Plus, that's not really an argument anyway because websites also frequently serve localised versions depending on the source IP, so the archive will at least sometimes not match what an individual user would see anyway. |
|
19:40
🔗
|
JAA |
Sure, access to an ArchiveBot pipeline can be shared. It's really just a script running on a machine in a tmux session. |
|
19:50
🔗
|
JAA |
tyzoid: However, we generally accept ArchiveBot pipelines only from people who have been around for a while since it's a long-term commitment. ArchiveBot pipelines need to be online continuously for months at a time. |
|
19:52
🔗
|
tyzoid |
JAA: I've not been around this project in particular for a long time, but I've been running several services for a while |
|
19:52
🔗
|
tyzoid |
https://status.arlm.tyzoid.com/ |
|
21:21
🔗
|
|
jschwart has quit IRC (Quit: Konversation terminated!) |
|
21:25
🔗
|
|
t2t2 has quit IRC (Ping timeout: 260 seconds) |
|
21:28
🔗
|
ivan |
archive.is appears to be lacking the cloudflare ddos filter or aggressive bot blocker right now |
|
21:30
🔗
|
|
schbirid has quit IRC (Quit: Leaving) |
|
21:32
🔗
|
|
t2t2 has joined #archiveteam-bs |
|
21:46
🔗
|
tyzoid |
JAA: Do you run a pipeline instance yourself? |
|
22:33
🔗
|
|
rbraun has quit IRC (Read error: Operation timed out) |
|
23:20
🔗
|
|
me_ is now known as yipdw |
|
23:23
🔗
|
|
rbraun has joined #archiveteam-bs |
|
23:37
🔗
|
|
BlueMax has joined #archiveteam-bs |