[00:40] *** justas1 has joined #archiveteam-ot [00:46] *** justas has quit IRC (Ping timeout: 745 seconds) [00:56] Lovely: a subtle behaviour of Tornado's IOLoop seems to have hidden a crash in ArchiveBot for several years. (For those interested: the pipeline reporting doesn't start immediately because the IOLoop doesn't run until the pipeline has attempted to get jobs from the control node. Although the callback for this is launched immediately, it doesn't run until the synchronous code lets the IOLoop run, or [00:56] something like that (didn't analyse it in detail, just assume it's similar to async/await). This is what caused new pipelines to show up as "(anonymous)" for a while. Among other things, the monitoring code checks the free disk space in the data directory. But that directory doesn't exist yet until the first item gets processed in seesaw. So the only reason this monitoring stuff didn't crash and burn [00:56] is because it indirectly delayed its execution until after the seesaw pipeline started running. And I only noticed this because I wanted to make it report to the control node immediately.) [01:03] Impatience begets patients [02:02] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [02:05] *** Verified_ has joined #archiveteam-ot [03:17] *** DogsRNice has quit IRC (Read error: Connection reset by peer) [04:00] *** lunik1 has quit IRC (:x) [04:02] *** lunik1 has joined #archiveteam-ot [04:34] *** nataraj has joined #archiveteam-ot [05:28] *** drcd_ has joined #archiveteam-ot [05:29] *** drcd has quit IRC (Ping timeout: 186 seconds) [05:29] *** drcd_ is now known as drcd [06:18] *** systwi has quit IRC (Read error: Connection reset by peer) [06:19] *** ShellyRol has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** Fusl__ has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** nyany_ has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** nyany has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** h3ndr1k has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** eientei95 has quit IRC (ircd.choopa.net irc.mzima.net) [06:19] *** svchfoo3 has quit IRC (ircd.choopa.net irc.mzima.net) [06:20] *** systwi has joined #archiveteam-ot [06:22] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [06:25] *** ShellyRol has joined #archiveteam-ot [06:25] *** Fusl__ has joined #archiveteam-ot [06:25] *** nyany_ has joined #archiveteam-ot [06:25] *** nyany has joined #archiveteam-ot [06:25] *** h3ndr1k has joined #archiveteam-ot [06:25] *** eientei95 has joined #archiveteam-ot [06:25] *** svchfoo3 has joined #archiveteam-ot [06:25] *** irc.mzima.net sets mode: +oo Fusl__ svchfoo3 [06:25] *** Fusl sets mode: +o Fusl__ [06:26] *** Fusl sets mode: +o svchfoo3 [06:26] *** Fusl_ sets mode: +o Fusl__ [06:26] *** Fusl_ sets mode: +o svchfoo3 [06:27] *** Fusl__ sets mode: +o kiska [06:27] *** Fusl__ sets mode: +o ivan_ [06:27] *** Fusl__ sets mode: +o kiska1 [06:27] *** Fusl__ sets mode: +o HCross [06:27] *** Fusl__ sets mode: +o hook54321 [06:27] *** Fusl__ sets mode: +o Kaz [06:27] *** Fusl__ sets mode: +o Fusl_ [06:27] *** Fusl__ sets mode: +o SketchCow [06:27] *** Fusl__ sets mode: +o AlsoJAA [06:27] *** Fusl__ sets mode: +o Fusl [06:27] *** Fusl__ sets mode: +o Kenshin [06:27] *** Fusl__ sets mode: +o dxrt [06:28] *** Fusl__ sets mode: +o kiskabak [06:28] *** Fusl__ sets mode: +o JAA [06:28] *** Fusl__ sets mode: +o jrwr [06:28] *** Fusl__ sets mode: +o astrid [06:28] *** Fusl__ sets mode: +o chfoo [06:28] *** Fusl__ sets mode: +o svchfoo1 [06:35] *** benjinsmi has joined #archiveteam-ot [06:37] *** benjins has quit IRC (Ping timeout: 252 seconds) [06:38] *** Verified_ has joined #archiveteam-ot [07:10] *** bluefoo has quit IRC (Read error: Operation timed out) [08:26] *** N4Y has quit IRC (Ping timeout: 745 seconds) [08:27] *** N4Y has joined #archiveteam-ot [08:34] *** bluefoo has joined #archiveteam-ot [09:07] *** Maylay has quit IRC (Ping timeout: 252 seconds) [09:08] *** Verified_ has quit IRC (Quit: Quit) [09:10] *** Maylay has joined #archiveteam-ot [09:29] *** icedice has joined #archiveteam-ot [09:47] I'm thinking about setting up borgbackup on a cheap Kimsufi (aka OVH) dedicated server [09:47] Is Kimsufi good enough for that? [10:32] yes but check out hetzner auction if you want more disk or gigabit [10:37] *** BlueMax has quit IRC (Quit: Leaving) [10:39] *** benjinss has joined #archiveteam-ot [10:41] *** benjinsmi has quit IRC (Ping timeout: 252 seconds) [10:49] *** bluefoo has quit IRC (Quit: bluefoo) [11:04] *** bluefoo has joined #archiveteam-ot [12:07] *** kiskabak has quit IRC (Remote host closed the connection) [12:07] *** kiskabak has joined #archiveteam-ot [12:08] *** Fusl sets mode: +o kiskabak [12:08] *** Fusl__ sets mode: +o kiskabak [12:08] *** Fusl_ sets mode: +o kiskabak [12:16] Not too sure about Hetzner [12:17] Germany seems like a terrible place to store copyrighted content [12:17] Though I guess full disk encryption could help with that [12:18] Hmm, they have servers in Finland nowadays as well [12:18] Tg [12:18] * though [12:19] I guess they'd still be somewhat under German jurisdiction [13:20] *** asie has joined #archiveteam-ot [13:55] *** killsushi has quit IRC (Quit: Leaving) [14:08] What content are you storing and where are you getting it from icedice ? Unless you get takedowns hetzner leave you alone, they won't go scanning your disk [14:11] *** mike__ has joined #archiveteam-ot [14:11] What do you need to do to sign up for an api key mike__ ? Could you sign up for a few ? [14:12] Hi all, I posted about this over in the main #archiveteam channel, but I'm trying to spread the word about a project to crowdsource legal data from Harvard Law Library. If you're interested in helping, we have a macOS app you can install or a docker image you can run. [14:12] The macOS app is here: https://apps.apple.com/us/app/legal-api-downloader/id1476586208?mt=12. The docker image instructions are here: https://free.law/legal-api-downloader/docker/ [14:13] The TOS of the API should be your guide, @Dallas: https://case.law/ [14:15] Dallas: Rare manga, cartoons from the 1990's and 2000's, some YouTube stuff, personal documents, nature photos that I've taken myself [14:16] Not sure if I'll bother with the more real TV series and movies [14:16] Those are higher risk and not as rare [14:18] The cartoons are mostly torrented, as with the TV series and movies [14:18] The manga is mostly DDL or collected by myself [14:19] The server wouldn't be publically accessible though [14:20] I can't obviously advocate piracy, but I can say that aside from torrents all those things will likely be fine for you to do with Hetzner icedice :) [14:20] I'll take a look mike__ [14:21] By not having the content easily available via legitimate means the copyright holders are encouraging piracy [14:23] Most people aren't as financially irresponsible as me that they import boxes filled with 15-20 year old manga magazines from Japan [14:24] Or as lucky that they found a anime dub on YouTube - that had only been broadcasted on TV and never available on DVD/streaming - before it was taken down [14:25] At least in the anime world, the copyright holders often don't know many older works of theirs exist [14:25] I don't disagree haha, I'm just saying for the record piracy is illegal etc. etc. but you won't have a prob on hetzner :D [14:25] Yeah [14:25] as in, they probably have it in the paperwork, they're just not very aware of them [14:26] I'd probably go with Kimsufi or Online.net just to be sure [14:26] Abuse reports sent to them about their baguette boxes go to dev/null [14:27] or "you've HEARD of this? nobody in Japan's heard of this!" kind of stuff [14:28] A manga artist started leaking her own old works on Twitter after she got tired of the publisher almost never publishing it in volumes and just letting it get lost in magazine land [14:28] Ryuutama's story is fascinating, it's a semi-notable Japanese traditional pen & paper RPG game [14:28] I think OVH are pretty ignorant of abuse reports but I'm sure I got one from online at some point, better plan, run any ... 'questionable', traffic through a vpn [14:28] the author had a print run with their publisher, but it ran out, and he got lots of requests to restock it [14:28] the publisher, however, refused to do another print run; the publisher also told him he's forbidden from putting it on competing "print-on-demand" services like Amazon's [14:29] the author then re-read his licensing agreement with the publisher, and ran it through a lawyer; apparently it was constructed in such a way that he only gave up the rights to *sell* the book [14:29] but could legally put it up for free... so he did [14:29] I and another person have imported all of her magazine exclusive works [14:29] hmm, or almost [14:29] Still one series that's not completely collected [14:30] And a bunch of magazine exclusive color pages that the artist herself has lost [14:30] I have a bunch of those as well [14:30] Another fascinating licensing story I always think of is when Tokyopop (an English manga licensor) C&D'd Kodansha (the Japanese manga publisher)... and won [14:30] (not in court, just, like, had a valid legal point) [14:30] Another artist wants to put her manga series in ebooks, but the publisher won't let her [14:33] Some manga I've managed to get have no record of existance on the Internet [14:33] So it'll be fun to scan those and surprise everyone [14:33] I have a Polish manga/anime fanzine whose sole record of existence on the Internet is a review on an equally obscure Polish website [14:38] *** DogsRNice has joined #archiveteam-ot [14:39] The manga I have was sold in connection with a tournament two decades ago [14:44] I and a few friends started a project where we imported English 41 manga volumes from a Southeast Asian publisher that went out of business years ago [14:45] We have a scanner with an automatic document feeder [14:45] https://www.youtube.com/watch?v=aaBLcgpXHZ4 [14:46] ^ This is an ADF scanner in action [14:46] Pretty excited about that [14:46] Had to import them from a secondhand marketplace via a forwarder [15:02] icedice: Noone in Germany (or anywhere else really) will care about your stored data as long as you don't distribute it. At least I've never heard of anything like that. Also, Borg has encryption anyway. [15:13] I guess upload filters won't apply to hosting providers and only to platforms [15:29] *** joepie91 has joined #archiveteam-ot [15:30] *** nataraj has quit IRC (Read error: Operation timed out) [15:46] *** bluefoo has quit IRC (Read error: Operation timed out) [16:32] *** icedice has quit IRC (Leaving) [17:13] *** mike__ has quit IRC (Quit: Page closed) [17:14] *** Arkiver2 has joined #archiveteam-ot [17:14] *** Fusl sets mode: +o Arkiver2 [17:14] *** Fusl__ sets mode: +o Arkiver2 [17:14] *** Fusl_ sets mode: +o Arkiver2 [17:18] *** Arkiver2 has quit IRC (Connection closed) [17:18] *** arkiver has joined #archiveteam-ot [17:18] *** Fusl__ sets mode: +o arkiver [17:18] *** Fusl sets mode: +o arkiver [17:18] *** Fusl_ sets mode: +o arkiver [17:45] *** Dragnog94 has quit IRC (The Lounge - https://thelounge.chat) [17:48] *** Dragnog94 has joined #archiveteam-ot [17:49] *** asdf0101 has quit IRC (Read error: Operation timed out) [17:49] *** markedL has quit IRC (Read error: Operation timed out) [18:52] *** ola_norsk has joined #archiveteam-ot [18:53] *** ola_norsk has quit IRC (leaving) [19:34] *** nataraj has joined #archiveteam-ot [19:46] I just saw for the first time how seesaw executes a pipeline, and I'm disgusted. [19:46] It reads the file contents and then exec()s the string. [19:47] Aer you really? [19:47] *are [19:47] I mean, There are worse things it could do. [19:47] And does :P [19:49] I can be disgusted at multiple things. :-P [20:13] *** antomatic has joined #archiveteam-ot [20:19] *** ShellyRol has quit IRC (Read error: Operation timed out) [20:20] *** ShellyRol has joined #archiveteam-ot [20:54] *** nataraj has quit IRC (Read error: Operation timed out) [21:16] my "merge millions (hundreds of thousands rather i think) of files into a deeply nested directory tree" finished in a day using a list from find as input for mv [21:16] instead of ETA of 1 week with rsync or mc [21:16] \o/ [22:34] *** BlueMax has joined #archiveteam-ot [23:38] *** markedL has joined #archiveteam-ot [23:38] *** asdf0101 has joined #archiveteam-ot [23:45] *** kiska1 has quit IRC (Remote host closed the connection) [23:45] *** kiska1 has joined #archiveteam-ot [23:45] *** Fusl__ sets mode: +o kiska1 [23:46] *** Fusl sets mode: +o kiska1 [23:46] *** Fusl_ sets mode: +o kiska1