#archiveteam 2019-06-20,Thu

↑back Search

Time Nickname Message
00:03 🔗 Ctrl has joined #archiveteam
00:04 🔗 MMovie has quit IRC (Quit: Leaving.)
00:11 🔗 yawkat has quit IRC (Ping timeout: 604 seconds)
00:22 🔗 godane has quit IRC (Ping timeout: 506 seconds)
00:23 🔗 Jens has quit IRC (Remote host closed the connection)
00:24 🔗 Jens has joined #archiveteam
00:29 🔗 yawkat has joined #archiveteam
00:34 🔗 Mayonaise has quit IRC (Read error: Operation timed out)
00:39 🔗 godane has joined #archiveteam
00:56 🔗 Ctrl has quit IRC (Ping timeout: 268 seconds)
01:02 🔗 Ctrl has joined #archiveteam
01:20 🔗 MMovie has joined #archiveteam
01:53 🔗 Ctrl has quit IRC (Max SendQ exceeded)
01:56 🔗 Ctrl has joined #archiveteam
02:08 🔗 wp494 has quit IRC (Ping timeout: 252 seconds)
03:44 🔗 kode54 has quit IRC (Remote host closed the connection)
03:44 🔗 odemg has quit IRC (Ping timeout: 265 seconds)
03:44 🔗 kode54 has joined #archiveteam
03:48 🔗 qw3rty117 has joined #archiveteam
03:52 🔗 qw3rty116 has quit IRC (Ping timeout: 600 seconds)
03:56 🔗 odemg has joined #archiveteam
04:07 🔗 Somebody2 Might be good to try and figure out a list of what is hosted on legacy.gitbook.com and grab it: https://twitter.com/GitBookStatus/status/1141437901907144706
04:08 🔗 Somebody2 It's the old version of a service, that went down for a while today.
04:08 🔗 Somebody2 It's not *that* high priority because all the stuff is also in git repos stored elsewhere, but it'd be nice to have in the Wayback Machine.
04:41 🔗 wyatt8740 has joined #archiveteam
05:22 🔗 kode54 has quit IRC (Quit: The Lounge - https://thelounge.chat)
05:24 🔗 kode54 has joined #archiveteam
05:34 🔗 spiff has joined #archiveteam
05:43 🔗 nataraj has joined #archiveteam
05:43 🔗 fuzzy8021 has quit IRC (Read error: Connection reset by peer)
05:45 🔗 fuzzy8021 has joined #archiveteam
06:15 🔗 tuluu has quit IRC (Ping timeout: 265 seconds)
06:17 🔗 tuluu has joined #archiveteam
06:22 🔗 deevious has joined #archiveteam
06:35 🔗 deevious has quit IRC (Quit: deevious)
06:52 🔗 deevious has joined #archiveteam
07:00 🔗 satoshi has quit IRC (Ping timeout: 265 seconds)
07:16 🔗 nataraj has quit IRC (Read error: Operation timed out)
07:26 🔗 nataraj has joined #archiveteam
07:57 🔗 stratum`_ has joined #archiveteam
07:57 🔗 cadbury has quit IRC (Ping timeout: 265 seconds)
07:57 🔗 spiff has quit IRC (Ping timeout: 265 seconds)
07:57 🔗 stratum` has quit IRC (Ping timeout: 265 seconds)
07:57 🔗 spifft has joined #archiveteam
07:57 🔗 spifft has quit IRC (Read error: Connection reset by peer)
07:57 🔗 Jon^ has quit IRC (Ping timeout: 265 seconds)
07:57 🔗 stratum`_ is now known as stratum`
07:57 🔗 jmtd has joined #archiveteam
07:58 🔗 burt_ has quit IRC (Ping timeout: 265 seconds)
07:58 🔗 spifft has joined #archiveteam
07:59 🔗 Laverne has quit IRC (Ping timeout: 265 seconds)
07:59 🔗 igloo251 has quit IRC (Ping timeout: 265 seconds)
08:00 🔗 omglolbah has quit IRC (Ping timeout: 265 seconds)
08:01 🔗 fuzzy8021 has quit IRC (Read error: Operation timed out)
08:01 🔗 mundus201 has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 fuzzy8021 has joined #archiveteam
08:02 🔗 edsu has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 edsu has joined #archiveteam
08:02 🔗 spifft has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 lukeman has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 wacky has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 TC01 has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 Gfy has quit IRC (Ping timeout: 265 seconds)
08:02 🔗 kiskabak has quit IRC (Ping timeout: 265 seconds)
08:03 🔗 omglolbah has joined #archiveteam
08:03 🔗 ats has quit IRC (Ping timeout: 265 seconds)
08:03 🔗 burt has joined #archiveteam
08:03 🔗 TC01 has joined #archiveteam
08:03 🔗 lukeman has joined #archiveteam
08:04 🔗 wacky has joined #archiveteam
08:04 🔗 ats has joined #archiveteam
08:09 🔗 mundus201 has joined #archiveteam
08:25 🔗 DLoader has joined #archiveteam
08:38 🔗 Mateon1 has quit IRC (Remote host closed the connection)
08:39 🔗 Mateon1 has joined #archiveteam
08:39 🔗 wacky has quit IRC (Read error: Operation timed out)
08:39 🔗 wacky_ has joined #archiveteam
08:39 🔗 Rondom has quit IRC (Read error: Operation timed out)
08:40 🔗 yawkat has quit IRC (Read error: Operation timed out)
08:41 🔗 nataraj has quit IRC (Read error: Operation timed out)
08:41 🔗 noirscape has quit IRC (Read error: Operation timed out)
08:41 🔗 brayden has quit IRC (Ping timeout: 268 seconds)
08:41 🔗 N4Y has quit IRC (Ping timeout: 268 seconds)
08:41 🔗 noirscape has joined #archiveteam
08:41 🔗 m007a83_ has joined #archiveteam
08:41 🔗 MrRadar2 has quit IRC (Read error: Operation timed out)
08:41 🔗 RichardG_ has joined #archiveteam
08:41 🔗 fennec has quit IRC (Read error: Operation timed out)
08:42 🔗 Ctrl has quit IRC (Ping timeout: 268 seconds)
08:42 🔗 Selanda_ has quit IRC (Ping timeout: 268 seconds)
08:42 🔗 randomdes has quit IRC (Ping timeout: 268 seconds)
08:42 🔗 bsmith093 has quit IRC (Ping timeout: 268 seconds)
08:42 🔗 fennec has joined #archiveteam
08:42 🔗 SmileyG has joined #archiveteam
08:42 🔗 oli has quit IRC (Read error: Operation timed out)
08:43 🔗 Hoolootwo has joined #archiveteam
08:43 🔗 af10b3e5e has joined #archiveteam
08:43 🔗 Selanda has joined #archiveteam
08:44 🔗 Xibalba has quit IRC (Ping timeout: 268 seconds)
08:44 🔗 odemgi_ has joined #archiveteam
08:44 🔗 nyaomi has quit IRC (Ping timeout: 268 seconds)
08:44 🔗 Rondom has joined #archiveteam
08:44 🔗 tsr has quit IRC (Ping timeout: 268 seconds)
08:44 🔗 BnAboyZ has quit IRC (Ping timeout: 268 seconds)
08:44 🔗 BnAboyZ has joined #archiveteam
08:44 🔗 Smiley has quit IRC (Ping timeout: 268 seconds)
08:44 🔗 voker57_ has quit IRC (Ping timeout: 365 seconds)
08:44 🔗 nataraj has joined #archiveteam
08:44 🔗 ats_ has joined #archiveteam
08:44 🔗 tsr has joined #archiveteam
08:45 🔗 Xibalba has joined #archiveteam
08:45 🔗 Fusl has quit IRC (Ping timeout: 365 seconds)
08:45 🔗 Wingy has quit IRC (Ping timeout: 268 seconds)
08:45 🔗 nyaomi has joined #archiveteam
08:45 🔗 ranma has quit IRC (Ping timeout: 252 seconds)
08:46 🔗 bsmith093 has joined #archiveteam
08:46 🔗 randomdes has joined #archiveteam
08:46 🔗 yawkat has joined #archiveteam
08:46 🔗 oli has joined #archiveteam
08:46 🔗 tuluu_ has joined #archiveteam
08:47 🔗 Coderjo_ has joined #archiveteam
08:47 🔗 noirscape has quit IRC (Ping timeout: 268 seconds)
08:47 🔗 Coderjo has quit IRC (Ping timeout: 365 seconds)
08:48 🔗 nataraj has quit IRC (Read error: Connection reset by peer)
08:48 🔗 colona_ has quit IRC (Ping timeout: 268 seconds)
08:48 🔗 noirscape has joined #archiveteam
08:49 🔗 nataraj has joined #archiveteam
08:49 🔗 Jeroen has quit IRC (Read error: Operation timed out)
08:49 🔗 BnAboyZ has quit IRC (Read error: Operation timed out)
08:49 🔗 brayden has joined #archiveteam
08:49 🔗 deevious has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 d5f4a3622 has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 Hooloovoo has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 RichardG has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 murmur has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 pew has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 ats has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 burt has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 tuluu has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 odemgi has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 seatsea has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 z00nx has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 kiska has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 joshua_ has quit IRC (Ping timeout: 268 seconds)
08:49 🔗 Fusl has joined #archiveteam
08:49 🔗 BnAboyZ has joined #archiveteam
08:49 🔗 Fusl_ sets mode: +o Fusl
08:49 🔗 jut has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 i0npulse has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 m007a83 has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 Flashfire has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 coderobe has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 Medo42 has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 lukeman has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 ColdIce has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 Nemo_bis has quit IRC (Ping timeout: 252 seconds)
08:49 🔗 Wingy has joined #archiveteam
08:49 🔗 colona has joined #archiveteam
08:50 🔗 joshua_ has joined #archiveteam
08:50 🔗 Xibalba has quit IRC (Ping timeout: 268 seconds)
08:50 🔗 z00nx has joined #archiveteam
08:50 🔗 kiska has joined #archiveteam
08:50 🔗 lukeman has joined #archiveteam
08:50 🔗 Flashfire has joined #archiveteam
08:50 🔗 jut has joined #archiveteam
08:50 🔗 ColdIce has joined #archiveteam
08:50 🔗 Medo42 has joined #archiveteam
08:50 🔗 coderobe has joined #archiveteam
08:50 🔗 ranma has joined #archiveteam
08:50 🔗 svchfoo3 sets mode: +o kiska
08:51 🔗 svchfoo1 sets mode: +v Flashfire
08:51 🔗 seatsea has joined #archiveteam
08:51 🔗 deevious has joined #archiveteam
08:51 🔗 Nemo_bis has joined #archiveteam
08:51 🔗 MrRadar2 has joined #archiveteam
08:52 🔗 Xibalba has joined #archiveteam
08:52 🔗 N4Y has joined #archiveteam
08:52 🔗 i0npulse has joined #archiveteam
08:52 🔗 pew has joined #archiveteam
08:52 🔗 svchfoo3 sets mode: +o MrRadar2
08:53 🔗 murmur has joined #archiveteam
08:55 🔗 Xibalba has quit IRC (Read error: Operation timed out)
08:55 🔗 voker57 has joined #archiveteam
08:57 🔗 Wingy has quit IRC (Ping timeout: 268 seconds)
08:57 🔗 Wingy has joined #archiveteam
08:58 🔗 Xibalba has joined #archiveteam
08:59 🔗 MrRadar2 has quit IRC (Remote host closed the connection)
08:59 🔗 MrRadar2 has joined #archiveteam
08:59 🔗 svchfoo1 sets mode: +o MrRadar2
08:59 🔗 colona has quit IRC (Ping timeout: 268 seconds)
08:59 🔗 treora has quit IRC (Ping timeout: 268 seconds)
09:00 🔗 treora has joined #archiveteam
09:01 🔗 Fusl has quit IRC (Ping timeout: 365 seconds)
09:01 🔗 Ganonmast has quit IRC (Ping timeout: 268 seconds)
09:01 🔗 burt has joined #archiveteam
09:02 🔗 colona has joined #archiveteam
09:05 🔗 BnAboyZ has quit IRC (Ping timeout: 268 seconds)
09:05 🔗 BnAboyZ has joined #archiveteam
09:05 🔗 spifft has joined #archiveteam
09:06 🔗 Gfy has joined #archiveteam
09:06 🔗 Laverne has joined #archiveteam
09:06 🔗 igloo251 has joined #archiveteam
09:07 🔗 kiskabak has joined #archiveteam
09:07 🔗 cadbury has joined #archiveteam
09:08 🔗 svchfoo3 sets mode: +o kiskabak
09:08 🔗 yuitimoth has quit IRC (Ping timeout: 268 seconds)
09:08 🔗 yuitimoth has joined #archiveteam
09:09 🔗 Tenebrae has quit IRC (Ping timeout: 365 seconds)
09:10 🔗 nataraj has quit IRC (Read error: Connection reset by peer)
09:10 🔗 nataraj has joined #archiveteam
09:11 🔗 Tenebrae has joined #archiveteam
09:16 🔗 Fusl has joined #archiveteam
09:16 🔗 Fusl_ sets mode: +o Fusl
09:20 🔗 Ganonmast has joined #archiveteam
09:24 🔗 BlueMax has quit IRC (Quit: Leaving)
09:48 🔗 Ctrl has joined #archiveteam
09:58 🔗 VerifiedJ has joined #archiveteam
09:58 🔗 Stilettoo has quit IRC (Read error: Operation timed out)
09:59 🔗 Stiletto has joined #archiveteam
10:41 🔗 deevious has quit IRC (Quit: deevious)
10:51 🔗 cfarquhar has quit IRC (Read error: Operation timed out)
10:52 🔗 cfarquhar has joined #archiveteam
11:44 🔗 wp494 has joined #archiveteam
12:03 🔗 Hani has quit IRC (Quit: Hani)
12:24 🔗 Odd0002_ has joined #archiveteam
12:30 🔗 Odd0002 has quit IRC (Read error: Operation timed out)
12:30 🔗 Odd0002_ is now known as Odd0002
12:47 🔗 deevious has joined #archiveteam
13:58 🔗 fredgido has quit IRC (Remote host closed the connection)
13:58 🔗 fredgido has joined #archiveteam
14:35 🔗 arbin has quit IRC (Ping timeout: 255 seconds)
14:35 🔗 arbin has joined #archiveteam
15:35 🔗 nataraj has quit IRC (Read error: Operation timed out)
15:35 🔗 spifft has quit IRC (Quit: spifft)
15:57 🔗 murmur has quit IRC (Quit: leaving)
15:58 🔗 murmur has joined #archiveteam
16:01 🔗 murmur has quit IRC (Client Quit)
16:27 🔗 murmur has joined #archiveteam
16:32 🔗 super3 has joined #archiveteam
16:32 🔗 super3 hello
16:39 🔗 dan| has joined #archiveteam
16:46 🔗 dan- has quit IRC (Ping timeout: 615 seconds)
16:52 🔗 super3 has quit IRC (Ping timeout: 260 seconds)
16:53 🔗 martini has joined #archiveteam
16:59 🔗 benjinsmi has joined #archiveteam
17:04 🔗 benjins has quit IRC (Read error: Operation timed out)
17:30 🔗 susudo has joined #archiveteam
17:47 🔗 m007a83_ is now known as m007a83
17:49 🔗 Mateon1 has quit IRC (Ping timeout: 360 seconds)
17:49 🔗 Mateon1 has joined #archiveteam
17:53 🔗 martini has quit IRC (Quit: No Reasson)
18:00 🔗 Jeroen has joined #archiveteam
18:00 🔗 spiff has joined #archiveteam
18:00 🔗 SketchCow So, we can classify this as interesting? interesting. FOS Archivebot channel is now backed up to the tune of 4 terabytes
18:01 🔗 SketchCow I checked, it's absolutely uploading
18:01 🔗 SketchCow Not sure what's going on - maybe an outage happened back before, maybe something else.
18:01 🔗 SketchCow But that's happening. I'm hoping it'll right itself.
18:05 🔗 Kaz S3 has been fairly shit recently, based on the moans I've seen in #newsgrabber
18:05 🔗 Kaz We're backed up to the tune of like 40TB ish iirc
18:05 🔗 Fusl 40TB?
18:05 🔗 Fusl i have like 100TB of newsgrabber stored
18:05 🔗 Fusl and yes, s3 is fairly shit recently
18:06 🔗 Fusl i'm getting a constant stream of 500 errors when uploading files
18:06 🔗 kiska And we have like 800TB of googleminus that we have to get moving as well
18:18 🔗 SketchCow I'll bring it up with operations
18:19 🔗 Fusl SketchCow: http://xor.meo.ws/0dd8abec/f3b9/4689/9333/97fe9808dcab.png no good and http://xor.meo.ws/e7ffd043/7cce/418c/985f/492734408995.png no good either
18:20 🔗 Fusl & absolutely no good: http://xor.meo.ws/d038a0fb/3139/4f42/993d/2626389c8dc4.png (shows how the throughput to the s3 interface is down to near zero since around two days)
18:21 🔗 Fusl so yeah, we're being backpressured by the s3 interface and therefore buffering a lot of data on archive team megawarc factory servers which we dont want
18:21 🔗 murmur has quit IRC (Quit: leaving)
18:22 🔗 murmur has joined #archiveteam
18:22 🔗 SketchCow Brought up
18:22 🔗 Fusl thanks
18:23 🔗 SketchCow "We had deployed a replacement kernel to mitigate the recent SACK linux vulnerabilities. Will roll that back (and instead rely on filtering for now to mitigate SACK)."
18:23 🔗 Kaz fwiw I think this is pre-SACK
18:23 🔗 Kaz but I guess every little helps
18:23 🔗 murmur has quit IRC (Client Quit)
18:24 🔗 Fusl exactly 46 hours is what i can tell from the graphs when everything turned upside down
18:25 🔗 murmur has joined #archiveteam
18:25 🔗 murmur has quit IRC (Client Quit)
18:26 🔗 murmur has joined #archiveteam
18:28 🔗 murmur has quit IRC (Client Quit)
18:31 🔗 murmur has joined #archiveteam
18:46 🔗 SketchCow They said that exactly matches two kernel upgrades
18:46 🔗 SketchCow They're reversing the easy one first
19:01 🔗 Fusl SketchCow: whatever they did, i guess the rollback, definitely increased the throughput by a lot again
19:01 🔗 Fusl we're back to similar levels as we were before, doing a 300-400 mbit/s to s3 now from abox-fks1 and 500ish mbit/s from abox-hel1
19:27 🔗 SketchCow I'll pass
19:28 🔗 SketchCow along
19:55 🔗 Igloo Oh is s3 faster now?
19:55 🔗 Fusl its as slow as two days ago but its faster than the last 48 hours :P
19:55 🔗 Igloo I'll take that as an improvement :P
20:01 🔗 SketchCow They're going to do more work on it
20:01 🔗 SketchCow But this at least pulls it back
20:02 🔗 Igloo Thank you for checking up on it :-)
20:19 🔗 SketchCow @textfiles - experimenting now with a much newer kernel on the s3 loadbalancer. again, will watch the stats but keep us posted...
20:20 🔗 Fusl SketchCow: can forward them this https://atdash.meo.ws/d/000000058/telegraf-detail?var-user=fusl&var-host=archivebox-hel1 and this https://atdash.meo.ws/d/000000058/telegraf-detail?var-user=fusl&var-host=archivebox-fks1 url
20:25 🔗 PhrackD has quit IRC (Read error: Operation timed out)
20:29 🔗 PhrackD has joined #archiveteam
20:36 🔗 murmur has quit IRC (Quit: leaving)
20:37 🔗 murmur has joined #archiveteam
21:36 🔗 squires has joined #archiveteam
21:56 🔗 susudo has quit IRC (Quit: Page closed)
22:52 🔗 MMovie has quit IRC (Quit: Leaving.)
22:58 🔗 balrog has quit IRC (Bye)
23:04 🔗 Ctrl has quit IRC (Ping timeout: 268 seconds)
23:24 🔗 VerifiedJ has quit IRC (Quit: Leaving)
23:40 🔗 ivan is now known as nothere
23:53 🔗 Ctrl has joined #archiveteam

irclogger-viewer