[00:24] *** DarkStar1 has quit IRC (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.) [00:34] *** kbtoo_ has quit IRC (Read error: Connection reset by peer) [00:36] *** kbtoo has joined #archiveteam-bs [00:40] *** Darkstar has joined #archiveteam-bs [00:40] *** wp494 has quit IRC (Read error: Operation timed out) [00:40] *** wp494 has joined #archiveteam-bs [00:52] *** julientm has joined #archiveteam-bs [01:47] *** Despatche has joined #archiveteam-bs [01:59] *** Exairnous has quit IRC (Ping timeout: 246 seconds) [02:03] *** fuzy802 has joined #archiveteam-bs [02:06] *** fuzzy8021 has quit IRC (Ping timeout: 252 seconds) [02:13] *** fuzy802 is now known as fuzzy8021 [02:14] *** SimpBrain has quit IRC (Read error: Connection reset by peer) [02:14] *** SimpBrain has joined #archiveteam-bs [02:28] *** user3434 has joined #archiveteam-bs [02:32] *** user3434 has quit IRC (Quit: Page closed) [02:36] *** PhrackD has quit IRC (Read error: Connection reset by peer) [02:41] *** PhrackD has joined #archiveteam-bs [02:44] *** Odd0002_ has joined #archiveteam-bs [02:48] *** Odd0002 has quit IRC (Ping timeout: 615 seconds) [02:48] *** Odd0002_ is now known as Odd0002 [03:02] *** SimpBrain has quit IRC (Read error: Connection reset by peer) [03:02] *** SimpBrain has joined #archiveteam-bs [03:41] *** julientm has quit IRC (Ping timeout: 615 seconds) [03:42] *** Despatche has quit IRC (Remote host closed the connection) [04:00] *** Stiletto has quit IRC (Ping timeout: 360 seconds) [04:00] *** Stilett0 has joined #archiveteam-bs [04:17] *** julientm has joined #archiveteam-bs [04:37] *** qw3rty117 has joined #archiveteam-bs [04:39] *** robogoat has quit IRC (Read error: Operation timed out) [04:43] *** qw3rty116 has quit IRC (Read error: Operation timed out) [04:46] *** robogoat has joined #archiveteam-bs [04:55] *** odemgi has quit IRC (Ping timeout: 246 seconds) [04:56] *** ndiddy has quit IRC () [04:58] *** odemgi has joined #archiveteam-bs [05:01] *** odemg has quit IRC (Ping timeout: 615 seconds) [05:08] *** odemg has joined #archiveteam-bs [05:43] *** Exairnous has joined #archiveteam-bs [05:53] *** bsmith093 has quit IRC (Ping timeout: 616 seconds) [05:53] *** julientm has quit IRC (Remote host closed the connection) [05:55] *** bsmith093 has joined #archiveteam-bs [05:59] *** kyonko has quit IRC (Read error: Connection reset by peer) [06:00] *** kyonko has joined #archiveteam-bs [06:05] *** SimpBrain has quit IRC (Remote host closed the connection) [06:06] *** SimpBrain has joined #archiveteam-bs [06:07] *** bsmith093 has quit IRC (Read error: Operation timed out) [06:10] *** bsmith093 has joined #archiveteam-bs [06:21] *** killsushi has quit IRC (Quit: Leaving) [06:25] *** deevious has joined #archiveteam-bs [06:35] *** SumTingWo has joined #archiveteam-bs [06:47] *** SumTingWo has quit IRC (Read error: Connection reset by peer) [06:51] *** julientm has joined #archiveteam-bs [06:55] *** julientre has joined #archiveteam-bs [07:08] *** Exairnous has quit IRC (Ping timeout: 255 seconds) [07:11] *** S1mpbrain has joined #archiveteam-bs [07:11] *** SimpBrain has quit IRC (Read error: Connection reset by peer) [07:13] *** julientre has quit IRC (Quit: Leaving) [07:17] *** julientm1 has joined #archiveteam-bs [07:24] *** julientm has quit IRC (Remote host closed the connection) [07:24] *** julientm1 has quit IRC (Leaving) [07:25] *** julientm has joined #archiveteam-bs [07:34] *** deevious has quit IRC (Quit: deevious) [08:11] *** deevious has joined #archiveteam-bs [08:40] *** Exairnous has joined #archiveteam-bs [09:33] *** julientm has quit IRC (Remote host closed the connection) [09:35] *** julientm1 has joined #archiveteam-bs [09:41] *** wp494 has quit IRC (Ping timeout: 492 seconds) [09:43] *** wp494 has joined #archiveteam-bs [09:52] *** julientm1 is now known as julientm [11:00] *** julientm has quit IRC (Remote host closed the connection) [11:29] *** kyonko has quit IRC (Read error: Connection reset by peer) [12:10] *** julientm has joined #archiveteam-bs [12:15] SketchCow: those pdfs of Spare Rib came back up, so I'm grabbing them now [12:22] *** julientm has quit IRC (Remote host closed the connection) [12:35] *** S1mpbrain has quit IRC (Remote host closed the connection) [12:35] *** S1mpbrain has joined #archiveteam-bs [13:13] *** deevious has quit IRC (Quit: deevious) [13:35] *** BlueMax has quit IRC (Read error: Connection reset by peer) [13:50] *** S1mpbrain has quit IRC (Read error: Operation timed out) [13:57] *** SimpBrain has joined #archiveteam-bs [14:27] *** deevious has joined #archiveteam-bs [15:08] *** ndiddy has joined #archiveteam-bs [15:21] *** deevious has quit IRC (Ping timeout: 252 seconds) [15:29] *** deevious has joined #archiveteam-bs [15:54] *** julientm has joined #archiveteam-bs [15:57] *** SimpBrain has quit IRC (Read error: Connection reset by peer) [15:58] *** SimpBrain has joined #archiveteam-bs [16:00] *** deevious1 has joined #archiveteam-bs [16:01] *** deevious has quit IRC (Read error: Connection reset by peer) [16:01] *** deevious1 is now known as deevious [16:09] *** julientm has quit IRC (Remote host closed the connection) [16:11] *** Czechball has joined #archiveteam-bs [16:11] So here I am [16:11] Czechball: Yes, it's possible to run all our projects outside of the warrior as well. [16:11] The instructions for that are normally in the code repository, which is linked on each project's wiki page. [16:11] yeah I noticed that [16:12] I thought maybe you have a Warrior distro ready for real hardware [16:12] There is a Docker image. [16:13] Alright, I'll check that out [16:13] I could probably even run some scripts on my RaspberryPi [16:24] *** godane has joined #archiveteam-bs [16:28] *** bitBaron has joined #archiveteam-bs [16:36] *** SimpBrain has quit IRC (Read error: Operation timed out) [16:40] *** SimpBrain has joined #archiveteam-bs [17:02] *** Stilett0 is now known as Stiletto [17:29] *** omarroth has joined #archiveteam-bs [17:39] *** julientm has joined #archiveteam-bs [17:40] *** Ravenloft has quit IRC (Read error: Connection reset by peer) [17:47] *** turnkit has joined #archiveteam-bs [17:56] *** julientm has quit IRC (Remote host closed the connection) [18:01] *** julientm has joined #archiveteam-bs [18:02] *** Moder112 has joined #archiveteam-bs [18:04] well [18:04] I tried splitting the megawarc I mentioned with those tools but unfortunately it resulted in the creation of either thousands of improperly named warc files [18:05] or a directory structure exactly the same as the original site but impossible to browse [18:06] I got the json metadata file for the archive, is it possible to split it into the files listed in the json? [18:06] or do I need to code a program from scratch to do that [18:17] *** bitBaron has quit IRC (Quit: My computer has gone to sleep. 😴😪ZZZzzz…) [18:33] I thought I could use this [18:33] https://github.com/alard/megawarc [18:33] but the tar file from the specific archive can't be downloaded [18:34] or I mean [18:34] It can be [18:34] but it saves as a 0b file [18:35] which I guess would make sense as the tar is for non-warc files [18:36] but the script really doesn't like that [18:40] *** wp494 has quit IRC (Read error: Operation timed out) [18:41] *** wp494 has joined #archiveteam-bs [18:43] *** Czechball has quit IRC (Leaving) [19:04] *** bitBaron has joined #archiveteam-bs [19:08] Moder112: I've never used it, but 'megawarc restore' should split the megawarc back up into the original files. But that'll write out all of the WARCs, not just the one(s) you want. [19:11] Moder112: Oh, forgot to mention, you can also only download the relevant part of the megawarc from IA with a range request. [19:13] *** Oddly has joined #archiveteam-bs [19:16] For example, to download the third WARC contained in https://archive.org/download/archiveteam_tumblr20181218030723 : [19:16] curl -L -H 'Range: bytes=46559344-46567731' https://archive.org/download/archiveteam_tumblr20181218030723/tumblr_20181218030723.megawarc.warc.gz > tumblr-tumblr-blog_obssesivebloodykisses-20181218-025918.warc.gz [19:17] The numbers in the range header come from the JSON record for that file: "offset":46559344,"size":8388 [19:17] The second number is offset + size - 1. [19:20] You could also extract that file from the megawarc using this: tail -c+46559345 tumblr_20181218030723.megawarc.warc.gz | head -c8388 > tumblr-tumblr-blog_obssesivebloodykisses-20181218-025918.warc.gz [19:20] Where the number for tail is the offset + 1 and the one for head is the size. [19:45] well [19:46] unfortunately I tried using the megawarc program [19:46] and it crashes on me [19:48] I'm gonna try out the partial download solution tomorrow [19:49] I'll come back to say how it went tomorrow [19:49] thanks a lot [20:00] *** omarroth has quit IRC (Ping timeout: 506 seconds) [20:01] *** Moder112 has quit IRC (Quit: Page closed) [20:01] *** Stilett0 has joined #archiveteam-bs [20:05] *** Stiletto has quit IRC (Ping timeout: 615 seconds) [20:22] *** kiska1 has quit IRC (Read error: Operation timed out) [20:23] *** kiska1 has joined #archiveteam-bs [20:31] *** kiska1 has quit IRC (Ping timeout (120 seconds)) [20:31] *** kiska1 has joined #archiveteam-bs [20:40] *** Despatche has joined #archiveteam-bs [20:54] lenary: Excellent. [20:55] SketchCow: how do I get the warcs somewhere safer than my spinning rust? [20:55] Just upload into IA's opensource collection, a script puts them in a safe place. [20:55] I haven’t attempted the other ephemera yet. Need to get Helena up and running [20:55] Oh neat [20:59] *** Oddly has quit IRC (Quit: Leaving) [21:00] *** BlueMax has joined #archiveteam-bs [21:42] *** BlueMax has quit IRC (Read error: Connection reset by peer) [21:43] *** SimpBrain has quit IRC (Remote host closed the connection) [21:47] *** SimpBrain has joined #archiveteam-bs [21:57] *** wyatt8740 has joined #archiveteam-bs [22:23] *** SimpBrain has quit IRC (Remote host closed the connection) [22:26] *** SimpBrain has joined #archiveteam-bs [23:00] *** Despatche has quit IRC (Quit: Connection reset by deer) [23:02] *** fuzzy8021 has quit IRC (Read error: Connection reset by peer) [23:03] *** fuzzy8021 has joined #archiveteam-bs [23:25] godane: I found a Hauppauge PVR-250 & an ASUS PVR-416 in my collection, and both seem to work