[00:01] *** Cameron_D has quit IRC (Read error: Operation timed out) [00:23] *** katocala has joined #archiveteam-bs [01:15] *** icedice has joined #archiveteam-bs [01:52] *** manjaro-u has quit IRC (Quit: Konversation terminated!) [01:54] tracker says: We're sorry, but something went wrong. [02:16] SketchCow: so i'm uploading both CNET TV and Techquickie youtube channel [02:17] i'm only doing webm format cause some problems with youtube have either missing fragments for the video only/audio only parts [02:27] *** godane has quit IRC (Read error: Connection reset by peer) [02:41] didn't know that youtube had missing frames or chunks from mp4 m4a [02:48] youtube is some kind of DASH container now [02:55] *** omglolba- has joined #archiveteam-bs [02:58] can confirm, i live inside it [03:04] *** omglolbah has quit IRC (Ping timeout: 745 seconds) [03:21] *** icedice has quit IRC (Quit: Leaving) [03:50] *** odemgi_ has joined #archiveteam-bs [03:52] *** odemgi has quit IRC (Ping timeout: 252 seconds) [03:57] *** qw3rty2 has joined #archiveteam-bs [04:01] *** SynMonger has quit IRC (Quit: Wait, what?) [04:02] *** SynMonger has joined #archiveteam-bs [04:02] *** qw3rty has quit IRC (Ping timeout: 745 seconds) [04:06] *** DogsRNice has quit IRC (Read error: Connection reset by peer) [04:10] *** RichardG_ has quit IRC (Read error: Operation timed out) [04:33] *** omglolbah has joined #archiveteam-bs [04:33] *** omglolba- has quit IRC (Ping timeout: 258 seconds) [04:38] *** wp494 has quit IRC (se.hub efnet.portlane.se) [04:38] *** zerkalo_ has quit IRC (se.hub efnet.portlane.se) [04:38] *** luckcolor has quit IRC (se.hub efnet.portlane.se) [04:38] *** dashcloud has quit IRC (se.hub efnet.portlane.se) [04:38] *** Sokar2 has quit IRC (se.hub efnet.portlane.se) [04:38] *** Deewiant has quit IRC (se.hub efnet.portlane.se) [04:38] *** Gfy has quit IRC (se.hub efnet.portlane.se) [04:38] *** Shen has quit IRC (se.hub efnet.portlane.se) [04:38] *** sHATNER has quit IRC (se.hub efnet.portlane.se) [04:38] *** Laverne has quit IRC (se.hub efnet.portlane.se) [04:38] *** closure has quit IRC (se.hub efnet.portlane.se) [04:39] *** wp494 has joined #archiveteam-bs [04:39] *** zerkalo_ has joined #archiveteam-bs [04:39] *** luckcolor has joined #archiveteam-bs [04:39] *** dashcloud has joined #archiveteam-bs [04:39] *** Sokar2 has joined #archiveteam-bs [04:39] *** Deewiant has joined #archiveteam-bs [04:39] *** Gfy has joined #archiveteam-bs [04:39] *** Shen has joined #archiveteam-bs [04:39] *** sHATNER has joined #archiveteam-bs [04:39] *** Laverne has joined #archiveteam-bs [04:39] *** closure has joined #archiveteam-bs [04:39] *** Deewiant has quit IRC (Ping timeout: 258 seconds) [04:39] *** Deewiant has joined #archiveteam-bs [04:42] *** zerkalo_ has quit IRC (Ping timeout: 258 seconds) [04:43] *** zerkalo has joined #archiveteam-bs [04:44] *** wp494 has quit IRC (Ping timeout: 258 seconds) [04:46] *** wp494 has joined #archiveteam-bs [04:47] *** Shen has quit IRC (Ping timeout: 258 seconds) [04:47] *** sHATNER has quit IRC (Ping timeout: 258 seconds) [04:48] *** Deewiant has quit IRC (Ping timeout: 258 seconds) [04:48] *** Sokar2 has quit IRC (Ping timeout: 258 seconds) [04:48] *** Gfy has quit IRC (Ping timeout: 258 seconds) [04:48] *** closure has quit IRC (Ping timeout: 258 seconds) [04:48] *** closure has joined #archiveteam-bs [04:48] *** luckcolor has quit IRC (Ping timeout: 258 seconds) [04:48] *** dashcloud has quit IRC (Ping timeout: 258 seconds) [04:48] *** Laverne has quit IRC (Ping timeout: 258 seconds) [04:49] *** dashcloud has joined #archiveteam-bs [04:50] *** luckcolor has joined #archiveteam-bs [04:53] *** sHATNER has joined #archiveteam-bs [04:58] so wait is youtube-dl considered not the best for downloading youtube videos [05:01] --abort-on-unavailable-fragment or patch it to make it the default [05:02] *** Deewiant has joined #archiveteam-bs [05:05] best sound/video quality? [05:09] i was doing bestvideo not webm and bestaudio not webm [05:18] xdax: seems the defaults (when i was using defaults) is to grab the mismatched mp4 + webm opus audio and shoehorn them into an MKV file for lack of a better container [05:19] you have to be very specific and dictate if you want webm/webm or if you want mp4/m4a [05:19] this is what i'm using now: [05:19] -f bestvideo[height<=?1080][ext=mp4]+bestaudio[ext=m4a]/bestvideo[height<=?1080][ext=mp4]+bestaudio[ext=mp3]/bestvideo[height<=?1080][ext=webm]+bestaudio[ext=webm]/best[height<=?1080]/bestvideo+bestaudio/best [05:45] i did close to 34 gigs of tv commercials recently from youtube now i'm wondering do i do them again but better [05:45] also is there a way to get the highest quality vimeo from an embedded vimeo MP4 [05:46] no access to the channel itself [05:51] *** RichardG has joined #archiveteam-bs [05:56] *** BlueMax has quit IRC (Read error: Connection reset by peer) [05:56] *** BlueMax has joined #archiveteam-bs [06:02] *** fredgido_ has joined #archiveteam-bs [06:05] *** Shen has joined #archiveteam-bs [06:05] *** Laverne has joined #archiveteam-bs [06:08] *** Gfy has joined #archiveteam-bs [06:09] *** fredgido has quit IRC (Read error: Operation timed out) [06:13] *** icedice has joined #archiveteam-bs [06:42] *** icedice2 has joined #archiveteam-bs [06:45] *** icedice has quit IRC (Ping timeout: 252 seconds) [06:56] *** icedice2 has quit IRC (Quit: Leaving) [08:36] *** MaximeleG has joined #archiveteam-bs [08:45] *** MaximeleG has quit IRC (Quit: MaximeleG) [08:54] *** Cameron_D has joined #archiveteam-bs [08:56] *** Dallas has joined #archiveteam-bs [09:42] *** BlueMax has quit IRC (Quit: Leaving) [09:43] *** BlueMax has joined #archiveteam-bs [10:04] *** Ryz has quit IRC (Remote host closed the connection) [10:04] *** kiska18 has quit IRC (Remote host closed the connection) [10:04] *** kiska18 has joined #archiveteam-bs [10:04] *** svchfoo1 sets mode: +o kiska18 [10:04] *** Fusl____ sets mode: +o kiska18 [10:04] *** Fusl sets mode: +o kiska18 [10:04] *** Fusl_ sets mode: +o kiska18 [10:04] *** Ryz has joined #archiveteam-bs [11:52] xdax: I'm assuming it's only available as an embedded video? did you try to put the embedded video URL into youtube-dl? [11:59] nvm, apparently it's not possible currently. https://github.com/ytdl-org/youtube-dl/issues/15763, there's an open PR for it though. [12:15] *** BlueMax has quit IRC (Read error: Connection reset by peer) [12:38] *** omglolbah has quit IRC (Read error: No route to host) [12:47] *** odemg has joined #archiveteam-bs [12:58] *** katocala has quit IRC () [13:00] *** omglolbah has joined #archiveteam-bs [13:25] *** systwi_ is now known as systwi [13:26] *** K4k has joined #archiveteam-bs [13:54] *** killsushi has joined #archiveteam-bs [14:11] *** tuluu has quit IRC (Remote host closed the connection) [14:13] *** tuluu has joined #archiveteam-bs [14:41] *** omglolbah has quit IRC (Read error: No route to host) [14:50] *** omglolbah has joined #archiveteam-bs [15:06] My picosong error processing is running now. This covers songs where any of the download failed previously, everything that happened while I was getting rate-limited, and a few items that didn't get fully processed due to qwarc dying after it ran out of memory. [15:10] Dash: By the way, my number yesterday wasn't quite right, the data prior to this error processing is 5.38 TiB. [15:41] *** godane has joined #archiveteam-bs [15:50] *** schbirid has joined #archiveteam-bs [15:55] *** DogsRNice has joined #archiveteam-bs [16:31] *** bithippo has joined #archiveteam-bs [16:33] *** bithippo is now known as toomuchto [16:34] *** omglolba- has joined #archiveteam-bs [16:43] *** omglolbah has quit IRC (Ping timeout: 745 seconds) [17:11] hook54321: i got the embedded videos but i'm wondering if they didn't upload in a higher quality [17:11] even though they're vhs->dvd->vimeo transfers [17:44] picosong finished earlier and yielded another ~66 GiB of WARCs. [17:44] JAA: awesome! [17:44] :D [17:45] Some songs seem to be completely broken on the server side, e.g. https://picosong.com/wmYLS/ always returns a 500. [17:45] that sucks [17:45] they probably lost some dat [17:45] didnt keep backups :P [17:45] Three or four cause the Disqus comments to fail to load because the song name is too long and causes a 414. [17:45] ISN'T DISQUS GREAT? [17:46] its just the best haha [17:46] Reminds me of the artist union [17:46] Here is the link to this audio file [17:47] Which is actually a PNG. [17:47] I still need to check a few things. I saw some 403s in the logs, for example. Need to sort those out. But basically it should be complete now. [17:47] ¯\_(ツ)_/¯ [17:51] Igloo: Doesn't that just mean that someone uploaded a PNG renamed to MP3 or whatever, and the server didn't bother to check whether it's actually an audio file? [17:51] I think actually JAA sometimes the website put the files into the wrong bucket. [17:52] Ah lol [17:52] I found some avatars which were MP3's [17:52] Nice [17:52] Yeah. [17:52] Broken on the website too so... [18:00] *** Flashfire has quit IRC (Remote host closed the connection) [18:00] *** kiska has quit IRC (Remote host closed the connection) [18:01] *** Flashfire has joined #archiveteam-bs [18:01] *** kiska has joined #archiveteam-bs [18:01] *** Fusl sets mode: +o kiska [18:01] *** Fusl_ sets mode: +o kiska [18:01] *** Fusl____ sets mode: +o kiska [18:06] *** kab0m has joined #archiveteam-bs [18:09] *** kab0m has left [18:20] i'm now at 1763k items [18:25] *** toomuchto has quit IRC (Textual IRC Client: www.textualapp.com) [19:07] *** RichardG has quit IRC (Read error: Operation timed out) [19:10] *** RichardG has joined #archiveteam-bs [19:47] I would like to see someone figure out the search keywords for important non-English YouTube content and !a everything in #youtubearchive [19:48] there are a lot of languages and many of these uploads have no English keywords [19:53] could we do something like page rank, that way we don't need to understand anything [19:56] I am happy with any approach that generates like 60%+ signal [20:22] *** killsushi has quit IRC (Connection closed) [20:28] *** killsushi has joined #archiveteam-bs [20:44] *** schbirid has quit IRC (Quit: Leaving) [21:48] *** BlueMax has joined #archiveteam-bs [21:53] *** Cameron_D has quit IRC (Quit: :(){ :|:& };:) [21:54] has anyone started archiving google fusion tables yet? [21:54] *** Cameron_D has joined #archiveteam-bs [22:16] is there a way to archive the results of a cfm search