[00:00] Boop. Did I miss anything [00:02] Regarding the discussion a few lines back, if you have to constantly start and shut your warrior, maybe you shouldn't be running a warrior. [00:10] does stopping and starting it frequently cause issues? [00:10] i leave mine running. just curious [00:13] Mayonaise: yes it does, if you don't let it gracefully stop [00:13] it means jobs get stuck [00:13] hrm evil [00:20] hi, is there any way to find a particular archived twitch vod? [00:21] I found a helpful searchable index, but its contents are not as broad as the contents of the github repo with a list of videos to grab [00:21] also it just has old twitch URLs which are not functional [00:24] did you check http://chfoo-cn.mooo.com/~archiveteam/twitchtv-index/html/ or the list here: http://archiveteam.org/index.php?title=Twitch.tv#What_we_are_saving [00:25] the former thing gives links to archives hosted by twitch? [00:26] its a rough index of what was saved based on the lists in the second link, AFAIK [00:26] I checked out the git repo, it says there is one archive saved for channel leveluplive2, but the former tool does not return any results for that channel [00:26] well I am not sure how to use the searchable index to find content that is not still hosted by twitch [00:29] It is not easy, we admit. [00:29] Analysis is one of my big pushes for 2015. [00:30] Hey, so swipnet is BASICALLY in the wayback machine. [00:30] which list was it on at the git repo [00:30] Next tie, we have to think of ways to do it that don't result in MASSIVE amounts of tiny files. [00:31] SketchCow: does FOS love you again? [00:31] It took the machine weeks and weeks to deal with that. [00:31] No, FOS is still pretty much manga torture comic land [00:32] But things are starting to finish up that are the most intense disk operations. [00:32] https://github.com/ArchiveTeam/twitchtv-items/blob/master/csv/highlights_top.csv in this file there is an entry for leveluplive2 [00:33] (also it is very weird for me to have a twitchtv-items repo next to all my other twitch repos) [00:33] I guess the csv may have been the input to a process that generated the much shorter https://github.com/ArchiveTeam/twitchtv-items/blob/master/items/video_pages/05_top_videos_10000views.txt, which is the list of stuff that was actually saved? [00:35] Like, it's still doing verizon files, also tiny, also legion. [00:36] Ha ha OOPS [00:36] I just opened all my screens on FOS and one of them is a move swipnet operation, still running. [00:36] It is officially past... 60 days. [00:37] That's the thing that just oves them to prepare them to be made into megawarcs. [00:37] ouch. [00:37] JUST moving. From one directory to another on the sae drive. [00:37] that should not take so long... what's the sheer number of files? [00:38] ha ha "should" [00:38] Justice does not reign in linux [00:38] No, our merry band of maniacs just put millions, millions of files into this filesystem. [00:38] It does not like. [00:38] sharpobje: what is the channel name again, please? [00:39] leveluplive2 [00:40] See, it has to do a "how big is this directory" after each file. [00:40] So it's doing that at the end, cand checking each pass multiple times. [00:40] This laptop is nice but the keyboard is shit. [00:46] sharpobje: sorry, I'm unable to find a video url. [00:46] alright, thanks [00:47] it looks like it was grabbed but I don't see a reference to the actual url for the video. [00:55] yeah, I can't figure out how to translate the highlight into the real url. [01:15] Verified and now deleting 3tb of Ancestry.com files. [01:15] they're all taken care of? [01:15] They're all uploaded. [01:15] I keep the original directory and the generated files until I'm sure it's all set. [01:16] aaaaaaaaa: the actual video urls contain the channel name, so if there is a list of those somewhere we could search it [01:17] https://github.com/ArchiveTeam/twitchtv-items/tree/master/items/flv_urls [01:17] but no liveuplive2 in any of them [01:18] tons of liveuplive, but no liveuplive2 [01:18] alright, thanks [01:18] sorry, leveluplive and leveluplive2 [01:19] but highlights use the video url of the original video, so if channel 1 makes a highlight of channel 2's video, the url would have channel 2's name in it [01:19] at least that is my understanding [01:27] FOS still a nightmare [01:29] FOS? [01:44] FOS is the magical machine that turns what we download into something usable by the internet archive [01:46] looks like this ted talk is as video/audio sync issue: https://archive.org/details/JaneMcGonigal_2010 [01:46] this is ted talk issue cause i have the problem with the original file and the 1500k file i have [02:07] My buddy Jane! [02:14] >___> [02:46] I can have more than one buddy Jane [06:35] http://www.infodisiac.com/Wikipedia/ScanMail/ is not much on wayback, archivebot please :) (no parent dirs) [12:35] http://www.apkmirror.com/ [12:41] i could do a brute force of the download links: http://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=970 [15:12] how to properly archive controversial Youtube videos that may be taken down because of political "censoring"? [15:19] Verified and now deleting 3tb of Ancestry.com files. [15:20] You do know we are still running? http://tracker.archiveteam.org/ancestry/ [15:35] I do. [15:35] That's 3tb of files that have been uploaded out of the hopper. [15:56] dserodio: youtube_dl --title --continue --retries 4 --write-info-json --write-description --write-thumbnail --write-annotations --all-subs --ignore-errors -f 38/138+141/138+22/138+140/138+139/264+141/264+22/264+140/264+139/137+141/137+22/137+140/137+139/37/22/135+141/135+22/135+140/135+139/best [15:56] https://github.com/ludios/youtube-dl [15:57] (needs to be on the wiki) [19:46] hi! can any of the admins look at github repository ownlog-grab? [19:46] I'm at the second round of testing those project - and so far everything is working fine [21:05] WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD [21:05] yahoosucks [21:05] what is your quest