#archiveteam-bs 2019-01-06,Sun

↑back Search

Time Nickname Message
00:02 🔗 ndiddy has joined #archiveteam-bs
00:11 🔗 wp494 has quit IRC (Read error: Operation timed out)
00:12 🔗 wp494 has joined #archiveteam-bs
00:22 🔗 Oddly has quit IRC (Ping timeout: 255 seconds)
00:34 🔗 BlueMax has joined #archiveteam-bs
01:02 🔗 step has quit IRC (ZNC 1.7.1 - https://znc.in)
01:13 🔗 hdch has quit IRC (Ping timeout: 265 seconds)
01:14 🔗 step has joined #archiveteam-bs
01:17 🔗 tomaspark has quit IRC (Ping timeout: 255 seconds)
01:19 🔗 Swiss- has joined #archiveteam-bs
01:24 🔗 tomaspark has joined #archiveteam-bs
01:28 🔗 hdch has joined #archiveteam-bs
01:53 🔗 Sanqui has quit IRC (Remote host closed the connection)
02:00 🔗 m007a83 has joined #archiveteam-bs
02:32 🔗 hdch has quit IRC (Ping timeout: 265 seconds)
02:46 🔗 JAA arkiver: Could you set up a target soon please? We need to get this started, only have roughly a day left anyway (or maybe a little more depending on when they're shutting it down, but there will be a massive number of job failures).
02:47 🔗 JAA Otherwise I suggest we just continue with the current target and sort out the mess afterwards. Should be easy enough to distinguish between the old and new files based on file modification date.
02:52 🔗 Sanqui has joined #archiveteam-bs
02:52 🔗 svchfoo3 sets mode: +o Sanqui
02:57 🔗 JAA Pinging some more people involved with targets: SketchCow HCross Kaz ^
03:18 🔗 kiska JAA: DId someone request a rsync target?
03:19 🔗 JAA kiska: That's also an option I think.
03:19 🔗 exoire has quit IRC (Read error: Operation timed out)
03:20 🔗 JAA The situation is that there's data from the old scripts on FOS, so continuing there would be messy. A fresh target would work.
03:26 🔗 kiska JAA: What is the item on IA?
03:28 🔗 phirephly JAA kiska speaking of rsync targets, would either of you be able to help with the newsgrabber rsync master being out of disk space or know who to ping about that?
03:28 🔗 kiska HCross: ^
03:30 🔗 JAA kiska: I don't think there is an item/collection yet.
03:30 🔗 kiska I see
03:30 🔗 JAA http://fos.textfiles.com/pipeline.html
03:31 🔗 JAA Mover script not running, and anyway only very little data so far because most thread IDs below 1 million don't exist.
03:39 🔗 MR9K has quit IRC (Quit: tilde lounge - https://irc.tilde.team)
04:06 🔗 kiska I'll leave rsync config until I have had breakfast... I borked up config
04:07 🔗 Flashfire kiska it is 3:07 pm
04:07 🔗 Flashfire Why have you not had breakfast
04:08 🔗 JAA That's not that bad. I had breakfast at around 19:00 today/yesterday.
04:10 🔗 JAA Actually no, later than that. Anyway, that's -ot material.
04:10 🔗 hdch has joined #archiveteam-bs
04:22 🔗 kiska JAA: How much space do you think I will need for the rsync target for the forums?
04:22 🔗 JAA kiska: No idea yet.
04:23 🔗 JAA We went through almost 50k thread IDs with only 4 GB of data, but that doesn't help at all because almost none of those threads exist. I intend to restart from ID 1 million, the threshold above which most threads appear to be.
04:24 🔗 JAA Uh no, 50k items with 10 thread IDs each, so 500k thread IDs.
04:24 🔗 JAA There are ~4.5 million IDs total.
04:32 🔗 qw3rty119 has joined #archiveteam-bs
04:36 🔗 qw3rty118 has quit IRC (Ping timeout: 600 seconds)
04:43 🔗 odemg has quit IRC (Ping timeout: 265 seconds)
04:55 🔗 odemg has joined #archiveteam-bs
05:04 🔗 hdch has quit IRC (Remote host closed the connection)
05:15 🔗 kiska JAA: I got the rsync target up on lightsail. rsync://rsync.kiska.pw/uol/:downloader/
05:15 🔗 JAA kiska: On Lightsail? How much capacity does that have?
05:18 🔗 kiska The $80/mo plan
05:19 🔗 kiska 16GB ram, 4 vCPU, 320GB ssd, 6TB transfer l
05:20 🔗 coldon2dr has quit IRC (Quit: https://quassel-irc.org - Chat comfortably. Anywhere.)
05:20 🔗 JAA Alright, we'll see how it goes. Thanks.
05:21 🔗 Kaz right
05:21 🔗 kiska I'll start setting up the megawarc factory and associated services
05:21 🔗 Kaz still need a target?
05:21 🔗 Kaz oh i see
05:25 🔗 JAA Adding 100k items from 1 million up now.
05:27 🔗 JAA And we're live. There will be many errors.
05:28 🔗 kiska JAA: Can you login to the rsync target? username is ubuntu
05:29 🔗 JAA kiska: Yup
05:29 🔗 kiska Can we keep window 0 for the megawarc processes?
05:31 🔗 JAA I don't mind how it's set up. I also have zero experience with targets and the megawarc factory.
05:31 🔗 kiska Neither do I xD
05:33 🔗 JAA So it looks like we're grabbing something.
05:33 🔗 JAA Very slow though.
05:34 🔗 JAA I did limit it to only 50 items per minute. On the other hand, it doesn't seem like the rate has anything to do with the errors.
05:34 🔗 Flashfire 10 200s then sleeping
05:35 🔗 JAA I got a lot of errors while testing as well. Perhaps other people are trying to download the forums as well.
05:35 🔗 Flashfire Does this project have its own channel yet?
05:35 🔗 JAA Nope, we didn't come up with a good name yet I think.
05:36 🔗 Flashfire maybe #UhOhL
05:36 🔗 JAA A channel for all of UOL, that is. The forums are just one part.
05:36 🔗 kiska Flashfire: lol
05:36 🔗 JAA Oh, I like that.
05:36 🔗 Flashfire is #UhOhL good
05:36 🔗 Flashfire I may not be able to code but stupid puns are my specialty
05:39 🔗 Albardin has joined #archiveteam-bs
05:39 🔗 w0rmybak has joined #archiveteam-bs
05:39 🔗 kiskabak has joined #archiveteam-bs
05:44 🔗 kiska SketchCow: When do I get a collection that I can upload UOL stuff into?
06:29 🔗 m007a83_ has joined #archiveteam-bs
06:32 🔗 m007a83 has quit IRC (Ping timeout: 252 seconds)
06:32 🔗 Swiss- has quit IRC (Read error: Connection reset by peer)
06:33 🔗 Swiss- has joined #archiveteam-bs
06:55 🔗 MR9K has joined #archiveteam-bs
07:19 🔗 Flashfire https://web.archive.org/save/ now offers saving of passworded stuff cool
07:31 🔗 Swiss- has quit IRC (Read error: Connection reset by peer)
07:48 🔗 ndiddy has quit IRC ()
07:52 🔗 MR9K has quit IRC (Quit: tilde lounge - https://irc.tilde.team)
07:55 🔗 MR9K has joined #archiveteam-bs
08:07 🔗 m007a83_ is now known as m007a83
08:13 🔗 MR9K has quit IRC (Quit: tilde lounge - https://irc.tilde.team)
08:13 🔗 MR9K has joined #archiveteam-bs
08:40 🔗 fredgido_ has joined #archiveteam-bs
08:47 🔗 fredgido has quit IRC (Read error: Operation timed out)
08:49 🔗 m007a83 has quit IRC (Read error: Connection reset by peer)
09:08 🔗 wp494 has quit IRC (Ping timeout: 265 seconds)
09:09 🔗 wp494 has joined #archiveteam-bs
09:27 🔗 m007a83 has joined #archiveteam-bs
09:28 🔗 Oddly has joined #archiveteam-bs
09:54 🔗 hook54321 has quit IRC (Quit: Connection closed for inactivity)
10:12 🔗 fredgido has joined #archiveteam-bs
10:13 🔗 fredgido_ has quit IRC (Ping timeout: 252 seconds)
11:42 🔗 hdch has joined #archiveteam-bs
11:54 🔗 BlueMax has quit IRC (Quit: Leaving)
12:09 🔗 Oddly has quit IRC (Read error: Operation timed out)
12:10 🔗 Oddly has joined #archiveteam-bs
13:49 🔗 Sian1468 has quit IRC ()
13:52 🔗 hdch has quit IRC (Quit: oops)
15:00 🔗 Mateon1 has quit IRC (Ping timeout: 360 seconds)
15:00 🔗 Mateon1 has joined #archiveteam-bs
16:24 🔗 hook54321 has joined #archiveteam-bs
16:25 🔗 Odd0002_ has joined #archiveteam-bs
16:26 🔗 Odd0002 has quit IRC (Ping timeout: 252 seconds)
16:26 🔗 Odd0002_ is now known as Odd0002
16:30 🔗 Oddly has quit IRC (Read error: Operation timed out)
16:39 🔗 exoire has joined #archiveteam-bs
16:40 🔗 exoire has quit IRC (Client Quit)
16:45 🔗 exoire has joined #archiveteam-bs
16:49 🔗 JAA XIVDB is offline. My grab didn't finish in time, but I can't tell you right now how much it managed to grab.
16:50 🔗 Terry has joined #archiveteam-bs
17:59 🔗 Terry is now known as NotGLaDOS
18:10 🔗 wp494 has quit IRC (Read error: Operation timed out)
18:10 🔗 wp494 has joined #archiveteam-bs
19:13 🔗 daan101 has joined #archiveteam-bs
19:21 🔗 Oddly has joined #archiveteam-bs
19:50 🔗 hook54321 has quit IRC ()
19:50 🔗 hook54321 has joined #archiveteam-bs
21:15 🔗 daan101 has quit IRC (Leaving)
21:26 🔗 ppsym has quit IRC (Remote host closed the connection)
21:31 🔗 brayden has quit IRC (Ping timeout: 260 seconds)
21:47 🔗 brayden has joined #archiveteam-bs
21:47 🔗 swebb sets mode: +o brayden
21:48 🔗 exoire has quit IRC (Read error: Operation timed out)
22:35 🔗 tomaspark has quit IRC (Read error: Connection reset by peer)
22:48 🔗 arkiver JAA: I see you fixed UOL :)
22:49 🔗 SimpBrain has joined #archiveteam-bs
22:51 🔗 BlueMax has joined #archiveteam-bs
23:08 🔗 Oddly has quit IRC (Ping timeout: 255 seconds)

irclogger-viewer