[00:45] *** Jens has quit IRC (Remote host closed the connection) [00:46] *** Jens has joined #urlteam [02:15] *** Kagee has quit IRC (Quit: WeeChat 2.5) [02:44] *** Kagee has joined #urlteam [03:37] *** odemg has quit IRC (Ping timeout: 265 seconds) [03:49] *** odemg has joined #urlteam [04:14] *** mtntmnky_ has joined #urlteam [04:16] *** mtntmnky has quit IRC (Remote host closed the connection) [17:32] Error communicating with tracker: 507 Server Error: The tracker needs an operator for manual maintenance. Try again later. for url: https://tracker.archiveteam.org:1338/api/get. [17:32] I think somebody needs to kick the tracker. [18:03] *** klg has quit IRC (Quit: brb) [18:05] Too many results! Run the export script. [18:05] Kaz [18:06] yes i know what that is [18:06] wait i wonder if that's this thing [18:08] *** klg has joined #urlteam [19:24] The export should run automatically...? [19:24] At least it normally does. [19:38] 2019-06-25 doesn't seem to have uploaded [19:39] which is, incidentally, when IA died [19:39] Ah [19:42] right, I'm running ~things~ [19:43] https://archive.org/details/urlteam_2019-06-25-04-17-08 [19:43] -> http://xor.meo.ws/SzCeNaJftUiGIK67TMMFxzoop6STjNd8/x.gif [19:45] well, that uploaded things but it hasn't fixed the tracker [19:46] poke it with a stick? [19:48] Do we need the one person who knows how all of this works? [19:48] it may clear automatically? [19:48] if Result.get_count() > DEADMAN_MAX_RESULTS: [19:48] return '
Too many results! Run the export script.
' [19:48] sec [19:50] it's running a backup job it seems [19:53] I'm letting it run.. if things don't pick up in an hour or so we'll poke someone with a brain [19:59] *** jut has joined #urlteam [20:06] okay had another look through, think I can see what happened [20:07] job failed to upload while IA was down, supervisor therefore didn't clear up after itself, so all jobs after that also failed [20:07] now running a drain/export & upload [20:08] https://tracker.archiveteam.org:1338/ this looks healthier, I guess [22:24] OSError: [Errno 28] No space left on device [22:24] oh [22:27] chfoo: could you have a look please? not sure what is safe to delete at this point.. I would assume ./export/done/ as it looks like that gets pushed elsewhere, but not sure if that'll get us as far as we need [22:33] the done directory is a backup directly after it gets uploaded to archive.org so it's safe to delete [22:33] directory* [22:34] did you run undrain script and delete the working export directory? [22:37] I was running export&upload, which is when it ran out of disk [22:37] so export folder currently has 1.9G in done (will delete), and 3.7G in 2019-06-27-etc [22:38] is current_working_set.txt in there safe to remove? Assuming that'll get recreated? [22:38] no, it's taken out of the database [22:38] ah, fun [22:39] so the undrain script puts the stuff back into the database [22:40] so undrain -> clear export/done/, run export&upload again? [22:41] clear export/done first to regain disk space, then undrain, and rerun [22:42] ok. I also ran a manual upload for a set from the other day (while IA was dead), that's still in the export folder, rather than /done/. Safe to delete? [22:42] yes, it should be ok [22:44] alternatively, you can download the working set data to a safe place, and just keep export & uploading until the results are cleared out, vacuum the database, then run the undrain script [22:46] Just kicked off undrain so will see how that goes, looks like we've had it running away a bit for the last few days https://usercontent.irccloud-cdn.com/file/t9Ywvhcj/image.png