[00:05] I think there's a spare -e in the "--env DOWNLOADER="your name" -e \" on https://github.com/ArchiveTeam/warrior-dockerfile [00:20] *** antomati_ has joined #archiveteam-bs [00:20] *** swebb sets mode: +o antomati_ [00:21] *** antomatic has quit IRC (Read error: Operation timed out) [00:25] *** antomati_ is now known as antomatic [00:37] *** arlen has joined #archiveteam-bs [00:38] *** Stilett0 has quit IRC () [00:39] SketchCow: is anyone archiving your twitch streams? [00:45] *** Stiletto has joined #archiveteam-bs [00:57] https://www.buzzfeednews.com/article/ryanhatesthis/france-paris-yellow-jackets-facebook [00:57] ^ The Facebook groups mentioned in this article would be very interesting to archive. Unfortunately, it'll be very difficult to find them. [00:58] *** adinbied has quit IRC (Read error: Operation timed out) [00:58] *** adinbied has joined #archiveteam-bs [00:58] *** arlen has quit IRC (arlen) [00:58] *** arlen has joined #archiveteam-bs [00:59] Well, some are linked directly, but there are certainly many, many more. [00:59] *** arlen has quit IRC (Client Quit) [01:25] *** wp494 has quit IRC (Ping timeout: 255 seconds) [01:25] *** wp494 has joined #archiveteam-bs [01:30] trvz: I hope not [01:32] lol why? [02:49] is there anyone that will run tubeup on random videos I find on my travels to the real obscure parts of the web [02:52] '*<-./:;<=>?@[\]`| [02:52] sorry. [02:52] :(){:|:&};: [02:53] .......... [02:53] At least when I copy and paste gibberish it makes a semblance of sense [02:54] My message makes perfect sense. [02:54] what is it...... [02:54] A fork bomb. [02:54] Anyway, this is -ot material. [02:55] on target [02:57] Flashfire: So what videos are we talking about here? And are they worth archiving? [02:57] most are. many of them lead to dead links [02:57] Its what happens when you dissect randomly downloaded files and google bits of them followed by FTP [02:58] you find some real obscure stuff [02:58] Open driveby downloads in textedit if they are cleared safe by virustotal [02:59] *** hdch has joined #archiveteam-bs [03:11] *** Mayonaise has quit IRC (Read error: Connection reset by peer) [03:15] *** arlen has joined #archiveteam-bs [03:18] *** Mayonaise has joined #archiveteam-bs [03:24] Great i lost most content i wanted to archive [03:29] *** Aoede has quit IRC (Ping timeout: 186 seconds) [03:30] *** VoynichCr has quit IRC (Read error: Operation timed out) [03:37] *** Aoede has joined #archiveteam-bs [03:38] *** svchfoo3 sets mode: +o Aoede [03:38] *** svchfoo1 sets mode: +o Aoede [03:40] *** VoynichCr has joined #archiveteam-bs [03:50] I created a git bundle of the uncloneable e30e/e98e GitHub repo which caused quite a stir the past few days: https://archive.org/details/github.com_e30e_e98e_git_bundle (Cc yipdw, nightpool, anarcat) [03:50] how'd you end up cloning it [03:50] or is bundling not the same operation [03:51] https://media.tenor.co/images/7f6402cf7df54cdf24505ee7895326aa/raw [03:51] Yes it is. :-) [03:51] I created a fork and renamed it. It was the repository name that caused it to be uncloneable (among all the other issues). [03:51] I mean, I know what bundle does [03:51] ah ok [04:06] *** qw3rty115 has joined #archiveteam-bs [04:11] *** qw3rty114 has quit IRC (Read error: Operation timed out) [04:14] *** odemgi_ has quit IRC (Read error: Operation timed out) [04:15] *** odemg has quit IRC (Ping timeout: 265 seconds) [04:15] *** odemgi has joined #archiveteam-bs [04:27] *** arlen has quit IRC (arlen) [04:27] *** odemg has joined #archiveteam-bs [05:02] *** DarkWorld has quit IRC (Read error: Connection reset by peer) [05:03] *** DarkWorld has joined #archiveteam-bs [05:11] *** arlen has joined #archiveteam-bs [05:24] JAA: Thanks for the clarification. [06:07] *** DarkWorld has quit IRC (Read error: Operation timed out) [06:43] *** schbirid has joined #archiveteam-bs [07:00] *** anarcat has quit IRC (Ping timeout: 265 seconds) [07:05] *** DFJustin has quit IRC (Remote host closed the connection) [07:11] *** DFJustin has joined #archiveteam-bs [07:11] *** swebb sets mode: +o DFJustin [07:19] *** Jusque has quit IRC (Quit: ZNC - http://znc.in) [07:20] *** Jusque has joined #archiveteam-bs [07:40] *** arlen has quit IRC (arlen) [08:04] *** DarkWorld has joined #archiveteam-bs [08:05] *** twoTBHetz has quit IRC (Read error: Operation timed out) [08:13] *** twoTBHetz has joined #archiveteam-bs [08:15] *** hdch has quit IRC (Quit: oops) [08:16] *** hdch has joined #archiveteam-bs [09:11] *** VerifiedJ has joined #archiveteam-bs [09:13] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [09:23] *** trc has joined #archiveteam-bs [09:24] *** DarkWorld has quit IRC (Read error: Connection reset by peer) [09:24] *** DarkWorld has joined #archiveteam-bs [09:48] *** jeekl has joined #archiveteam-bs [09:48] *** BlueMax has quit IRC (Quit: Leaving) [09:49] *** fireglow has joined #archiveteam-bs [09:49] Brewtoad.com, a community for home-brewers is shutting down at the end of the month. Lots of community made home brew recipes there. [10:17] *** trc has quit IRC (Remote host closed the connection) [10:20] *** antomati_ has joined #archiveteam-bs [10:20] *** swebb sets mode: +o antomati_ [10:21] *** adinbied has quit IRC (Read error: Connection reset by peer) [10:21] *** antomatic has quit IRC (Read error: Operation timed out) [10:21] *** antomati_ is now known as antomatic [10:22] *** adinbied has joined #archiveteam-bs [10:28] *** wp494 has quit IRC (Ping timeout: 506 seconds) [10:28] *** wp494 has joined #archiveteam-bs [10:47] *** hdch has quit IRC (Remote host closed the connection) [10:47] *** _hdch_ has joined #archiveteam-bs [12:10] *** caff has quit IRC (Read error: Connection reset by peer) [12:52] *** anarcat has joined #archiveteam-bs [12:57] *** DarkWorld has quit IRC (Ping timeout: 600 seconds) [12:57] *** DarkWorld has joined #archiveteam-bs [13:26] https://github.com/wallabag/design/blob/master/archetypes/archivist.svg [14:21] *** DarkWorld has quit IRC (Ping timeout: 633 seconds) [15:26] *** DFJustin has quit IRC (Remote host closed the connection) [15:27] *** DFJustin has joined #archiveteam-bs [15:27] *** swebb sets mode: +o DFJustin [16:10] jrwr: Looks like the bot updating the default warrior project on the wiki broke. [16:34] *** Verified_ has joined #archiveteam-bs [16:36] *** VerifiedJ has quit IRC (Ping timeout: 252 seconds) [16:42] *** Boppen has quit IRC (Quit: Nettalk6 - www.ntalk.de) [17:19] *** _hdch_ has quit IRC (Remote host closed the connection) [17:19] *** _hdch_ has joined #archiveteam-bs [17:29] *** twoTBHetz has quit IRC (Read error: Operation timed out) [17:32] jeekl: I had three competing definitions of 'home-brewers' in my head there :) [17:44] *** etudier has joined #archiveteam-bs [17:45] *** etudier is now known as jesso [17:51] *** benjinsmi has quit IRC (Leaving) [17:52] *** benjins has joined #archiveteam-bs [17:59] *** twoTBHetz has joined #archiveteam-bs [18:02] Thanks Nemo bis. [18:03] *** jesso_ has joined #archiveteam-bs [18:06] *** lindalap has joined #archiveteam-bs [18:09] Archival request: https://twitter.com/JagexAyiza/ and https://twitter.com/AyizaTV [18:10] "This is likely the last tweet you’ll see from this account as I leave Jagex on Friday." https://twitter.com/JagexAyiza/status/1072912102065692672 [18:25] *** jesso has quit IRC (Read error: Operation timed out) [19:09] *** jut has quit IRC (Quit: Ram upgrade has arrived. Woot!!) [19:21] *** caff has joined #archiveteam-bs [19:25] *** wp494 has quit IRC (Ping timeout: 255 seconds) [19:26] *** wp494 has joined #archiveteam-bs [19:29] *** _hdch_ has quit IRC (Quit: oops) [19:29] *** twoTBHetz has quit IRC (Read error: Operation timed out) [19:39] *** twoTBHetz has joined #archiveteam-bs [20:03] *** twoTBHetz has quit IRC (Ping timeout: 1212 seconds) [20:07] *** jut has joined #archiveteam-bs [20:08] so i think i will use jpg vs png for screenshots with zinio [20:09] mostly cause you would be getting 100mb pdfs that look like crap and the jpg screenshots look fine [20:12] *** twoTBHetz has joined #archiveteam-bs [20:14] *** keith20 has joined #archiveteam-bs [20:16] *** keith20 has quit IRC (Remote host closed the connection) [20:16] *** keith20 has joined #archiveteam-bs [20:19] *** keith20 has quit IRC (Client Quit) [20:20] joepie91: hehe, yeah, sorry should've been more specific :-) [20:35] *** Dimtree has quit IRC (Peace) [20:39] *** Dimtree has joined #archiveteam-bs [20:56] *** Mateon1 has quit IRC (Ping timeout: 265 seconds) [20:56] *** Mateon1 has joined #archiveteam-bs [21:26] *** BlueMax has joined #archiveteam-bs [21:27] *** VerifiedJ has joined #archiveteam-bs [21:29] *** Verified_ has quit IRC (Ping timeout: 252 seconds) [21:57] *** lindalap has quit IRC (Quit: lindalap) [22:18] *** melissa has joined #archiveteam-bs [22:18] *** simone2 has joined #archiveteam-bs [22:19] hi Kaz [22:19] eyy. Blackberry World is going to shut down. Could you guys back up all the apps on there? [22:19] Argh! [22:19] o/ [22:19] simone2: join #tumbledown [22:19] *** kiska was kicked by Kaz (sleep) [22:19] *** Kaz sets mode: +b kiska!*@* [22:19] hi trvz [22:19] *** simone2 is now known as chauffer [22:20] uh, unless you want to do all projects... [22:20] so how many warriors do i run per host [22:20] Kaz, [22:20] really depends on your host [22:20] (and which project) [22:21] is there any IP limitations? [22:21] how much cpu/ram/etc? [22:21] like archiveteam ratelimiting by IP [22:21] and most importantly, disk [22:21] none on our end [22:21] okay [22:21] any reason why concurrent_items is capped to 6 in the code? [22:21] if you have an epyc server with 256GB+ RAM, you'll get rate limited by tumblr [22:21] ah [22:21] because warriors tend to be fairly low powered [22:22] trvz: not if you have enough IPs.. [22:22] who can be bothered to set up multiple IPs tho [22:22] easier to run more servers [22:22] so i'm prolly ratelimited by tumblr i guess [22:22] I mean, if you're blowing however much money of a 256gb ram server.. a few minutes for a few IPs won't kill you [22:22] melissa: Any date? [22:22] chauffer: what specs have you got for your warrior [22:23] dual e5 64gb [22:23] run 120 jobs at once then [22:23] Is blackberryworld a job for archivebot or? [22:23] and disk? [22:23] some tbs hdd [22:23] Sounds like it would be a better warrior project [22:23] raid5-6 [22:23] JAA, December 31, 2019 [22:23] cool [22:23] go nuts then [22:23] ~200 threads [22:23] ya but i guess tumblr is ratelimiting me no? [22:24] check log? [22:24] Who cares just do what you can chauffer [22:24] i can only do 6 concurrent items & 20 rsync [22:24] should be code 429 [22:24] github.com/archiveteam/tumblr-grab [22:24] are they in stdout? [22:24] stop using the warrior, start running the scripts manually [22:24] ah [22:24] nice [22:24] is tthere a docker image for this [22:25] no [22:25] no good [22:25] just use debian or something [22:25] noo... [22:25] Hehehehe [22:25] "or something" [22:25] docker only sry [22:25] well [22:26] run debian in docker? [22:26] i will make an image [22:26] sounds like you want to spin up a good chunk of warriors then [22:26] I wouldn't like to be the one trying to manually manage them though [22:26] go big or go home [22:26] also go kubernetes [22:26] Just run the one warrior and hope it downloads fast? [22:26] are you a troll Flashfire [22:27] No I am not. I’m just a tech noob [22:27] If I was a troll they would have gotten rid of me by now [22:27] okay, so i'm afraid it's not that easy :D [22:27] I'm interested [22:27] why not [22:28] why not what [22:28] why is debian off the table [22:28] why are you limited to docker [22:28] i just don't run things not in containers [22:29] well, you're running it in screen [22:29] https://hub.docker.com/_/debian/ i guess [22:29] ho ho ho 🎅 [22:29] does that count? [22:30] *** Kaz sets mode: -b kiska!*@* [22:39] *** t2t2 has quit IRC (Read error: Operation timed out) [22:39] *** t2t2 has joined #archiveteam-bs [22:45] *** DarkWorld has joined #archiveteam-bs [22:58] *** tuluu has quit IRC (Remote host closed the connection) [23:02] What was the issue with Debian? [23:02] *** tuluu has joined #archiveteam-bs [23:02] I'm not sure what happened. [23:03] Oh. [23:05] The VirtualBox VM is not powerful enough. That's why it's limited to 6 concurrent jobs. It is set to prevent the VM from freezing. [23:12] *** RichardG has quit IRC (Read error: Connection reset by peer) [23:14] *** RichardG has joined #archiveteam-bs [23:14] *** victorbje has quit IRC () [23:15] *** diggan has joined #archiveteam-bs [23:16] *** DarkWorld has quit IRC (Ping timeout: 600 seconds) [23:17] *** DarkWorld has joined #archiveteam-bs [23:20] *** Atom-- has joined #archiveteam-bs [23:22] *** Atom__ has quit IRC (Read error: Operation timed out) [23:25] docker run -d chauffer/trashcan:tumbledown [23:25] better than installing All That Shit [23:26] JAA, Flashfire so what d'you think? [23:26] December 31, 2019 shut down of BB world [23:33] melissa: I created a wiki page for it at least. It looks like downloads require an account or are at least not directly accessible. If you know anything about how the thing works, feel free to dump it here. Fortunately, we have over a year to figure it out. [23:35] I have an couple of BB devices from which they can be downloaded. [23:35] *** kpcyrd has joined #archiveteam-bs [23:36] Cool. Any chance you could try figuring out *how* it downloads the apps? [23:37] I have zero experience with BlackBerry, and nearly zero with debugging/reverse-engineering mobile phones and similar devices in general. [23:45] I have never done it. Maybe with guidance? [23:52] https://forums.crackberry.com/blackberry-10-apps-f274/releasing-bar-files-after-bbworld-goes-down-1134699/ related