[00:01] *** Despatche has joined #archiveteam-bs [00:30] *** Sk1d has quit IRC (Read error: Operation timed out) [00:33] *** Sk1d has joined #archiveteam-bs [00:56] *** godane has quit IRC (Read error: Operation timed out) [01:06] *** Despatche has quit IRC (Remote host closed the connection) [01:16] *** Despatche has joined #archiveteam-bs [01:16] *** Despatche has quit IRC (Read error: Connection reset by peer) [01:17] *** Despatche has joined #archiveteam-bs [01:30] *** Despatche has quit IRC (Quit: Connection reset by deer) [02:07] *** m007a83 has quit IRC (Read error: Connection reset by peer) [02:08] *** HashbangI has quit IRC (Read error: Operation timed out) [02:10] *** m007a83 has joined #archiveteam-bs [02:24] *** HashbangI has joined #archiveteam-bs [02:45] *** BlueMax has quit IRC (Read error: Connection reset by peer) [02:51] *** Sk1d has quit IRC (Read error: Operation timed out) [02:56] *** Sk1d has joined #archiveteam-bs [03:01] *** Sk1d has quit IRC (Read error: Operation timed out) [03:04] *** Sk1d has joined #archiveteam-bs [03:14] *** Sk1d has quit IRC (Read error: Operation timed out) [03:16] *** Sk1d has joined #archiveteam-bs [03:23] *** icedice has quit IRC (Quit: Leaving) [03:28] *** Sk1d has quit IRC (Read error: Operation timed out) [03:32] *** Sk1d has joined #archiveteam-bs [03:40] jrwr: I saw, independently before I came in here, that you'd fixed it. Thank you very much. [03:40] Now, let's document. [04:05] https://internetarchive.archiveteam.org/index.php?title=300_Funston_Avenue [04:06] If people want to edit on the archive and add requests in the war room for me to add essays as they occur to me, that's always welcome. [04:15] *** BlueMax has joined #archiveteam-bs [04:17] *** odemgi_ has joined #archiveteam-bs [04:23] *** odemgi has quit IRC (Read error: Operation timed out) [04:24] *** ndiddy has quit IRC (Ping timeout: 252 seconds) [04:26] *** odemg has quit IRC (Ping timeout: 615 seconds) [04:29] *** godane has joined #archiveteam-bs [04:51] *** Sk1d has quit IRC (Read error: Operation timed out) [04:54] *** Sk1d has joined #archiveteam-bs [04:57] *** qw3rty115 has joined #archiveteam-bs [05:02] *** qw3rty114 has quit IRC (Read error: Operation timed out) [05:04] *** Mateon1 has quit IRC (Ping timeout: 268 seconds) [05:04] *** Mateon1 has joined #archiveteam-bs [05:16] *** wp494 has quit IRC (Ping timeout: 506 seconds) [05:16] *** wp494 has joined #archiveteam-bs [06:21] *** Sk1d has quit IRC (Read error: Operation timed out) [06:24] *** Sk1d has joined #archiveteam-bs [07:02] *** Exairnous has quit IRC (Read error: Operation timed out) [07:23] *** wyatt8750 has quit IRC (Ping timeout: 360 seconds) [07:25] *** Sk1d has quit IRC (Read error: Operation timed out) [07:28] *** Sk1d has joined #archiveteam-bs [07:53] *** wyatt8740 has joined #archiveteam-bs [08:04] *** PurpleSym sets mode: +o SketchCow [08:11] *** Sk1d has quit IRC (Read error: Operation timed out) [08:14] *** Sk1d has joined #archiveteam-bs [09:03] *** wyatt8740 has quit IRC (Ping timeout: 255 seconds) [09:14] *** Mateon1 has quit IRC (west.us.hub irc.Prison.NET) [09:14] *** Polylith has quit IRC (west.us.hub irc.Prison.NET) [09:14] *** SynMonger has quit IRC (west.us.hub irc.Prison.NET) [09:14] *** chirlu has quit IRC (west.us.hub irc.Prison.NET) [09:14] *** achip has quit IRC (west.us.hub irc.Prison.NET) [09:14] *** marked has quit IRC (west.us.hub irc.Prison.NET) [09:16] *** Polylith_ has joined #archiveteam-bs [09:18] *** synm0nger has joined #archiveteam-bs [09:19] *** Sk1d has quit IRC (Read error: Operation timed out) [09:23] *** chirlu` has joined #archiveteam-bs [09:23] *** Sk1d has joined #archiveteam-bs [09:25] *** chirlu has joined #archiveteam-bs [09:25] *** chirlu has quit IRC (Ping timeout: 255 seconds) [09:25] *** marked has joined #archiveteam-bs [09:26] *** achip has joined #archiveteam-bs [09:30] *** Mateon1 has joined #archiveteam-bs [09:39] *** Sk1d has quit IRC (Read error: Operation timed out) [09:43] *** Sk1d has joined #archiveteam-bs [10:19] The YouTube comment downloader is ready for testing. [10:19] Download it here: /home/user/Documents/youtube_comments/no_polymer/downloader/documentation.txt [10:19] Woops wrong link. [10:19] Get it from here: http://163.172.39.176/youtube_comments_downloader/documentation.txt [10:19] Still wrong link: http://163.172.39.176/youtube_comments_downloader/ [10:19] The most recent one is the correct one. [10:20] Download all the files to run it. [10:20] Note that you need Python 3 and the following third party libraries: [10:20] esprima, requests, BeautifulSoup, lxml [10:21] esprima might not be avaliable via your distro's package manager. If that is the case get it with pip. [10:22] Also it can accept either just the video IDs or an entire link as long as it starts with http:// or https://. [10:22] The link must also contain youtube.com. [10:22] The reason for these requirments is so we don't accidently treat a raw video ID as a link. [10:22] Please test the program and provide feedback. [10:23] I am interested in knowing the following: [10:23] 1. Did it get all of the comments on the video you tried (including replies)? [10:23] 2. Did it work in your country? [10:24] The reason I ask #2 is because I know YouTube has regional versions of the site that they automaticlly redirect to based on IP address. [10:25] 3. Is there any missing information from the comments JSON that should be added? [10:25] Although I think I got everything maybe there is a video where a comment uses some kind of special feature that I did not know about and it requires more code to save it. [10:28] Also please ask any questions that you might have about this. [10:30] *** Sk1d has quit IRC (Read error: Operation timed out) [10:32] *** Sk1d has joined #archiveteam-bs [10:37] I should have mentioned: to run the program do: python3 ./youtube_comments.py --log-response -- [video IDs/URLs go here]. [10:38] The log-response flag is optional but it is good to have enabled so if we have an issue we can troubleshoot it easier. [10:54] *** Despatche has joined #archiveteam-bs [11:00] *** Despatche has quit IRC (Quit: Connection reset by deer) [11:06] *** odemgi has joined #archiveteam-bs [11:08] *** odemgi_ has quit IRC (Read error: Operation timed out) [11:19] Oh, first time I hear about esprima, looks interesting. [11:51] *** omglolbah has quit IRC (Read error: Operation timed out) [12:15] *** BartoCH_ has joined #archiveteam-bs [12:15] *** BartoCH has quit IRC (Read error: Connection reset by peer) [13:05] *** PurpleSym has quit IRC (Quit: *) [13:05] *** PurpleSym has joined #archiveteam-bs [13:05] *** svchfoo1 sets mode: +o PurpleSym [13:06] *** BlueMax has quit IRC (Read error: Connection reset by peer) [14:08] *** Oddly has joined #archiveteam-bs [14:12] *** wp494 has quit IRC (Read error: Operation timed out) [14:12] *** wp494 has joined #archiveteam-bs [14:30] *** schbirid has joined #archiveteam-bs [14:50] *** BartoCH_ is now known as BartoCH [14:53] *** Sk1d has quit IRC (Read error: Operation timed out) [14:55] *** Sk1d has joined #archiveteam-bs [16:19] *** Sk1d has quit IRC (Read error: Operation timed out) [16:21] *** Sk1d has joined #archiveteam-bs [17:09] I got a 503 while testing, so I updated it to increase the delays. I also added an Accept-Language header since I noticed that it was being sent by Firefox. [17:09] We will almost certainly need to run this with a concurency of one. [17:11] Also one thing I am thinking of: how will we check the results that the clients send to the server? Maybe we keep a hash of the IP addresses that send the results and then ban clients if they send bad data. [17:12] The problem is checking the results is that someone may add, remove or edit a comment during the timestamp between the first time we fetch the comments and the second time. [17:12] Whatever algorithm we use to check the results will need to take this into account. [17:12] * The problem with checking .. [17:13] * during the time between [17:13] *** Terbium has quit IRC (Ping timeout: 360 seconds) [17:15] I should have specified the update is avaliable via the same URL: http://163.172.39.176/youtube_comments_downloader/ [17:15] Please redownload. [17:16] *** icedice has joined #archiveteam-bs [17:34] *** tomaspark has quit IRC (Read error: Operation timed out) [17:34] *** tomaspark has joined #archiveteam-bs [17:43] *** Terbium has joined #archiveteam-bs [17:48] *** tomaspark has quit IRC (Read error: Connection reset by peer) [17:48] *** Sk1d has quit IRC (Read error: Operation timed out) [17:49] *** tomaspark has joined #archiveteam-bs [17:51] *** Sk1d has joined #archiveteam-bs [18:16] *** Sk1d has quit IRC (Read error: Operation timed out) [18:18] *** Sk1d has joined #archiveteam-bs [18:22] *** balrog has quit IRC (Read error: Operation timed out) [18:26] *** chimyatta has joined #archiveteam-bs [18:29] *** balrog has joined #archiveteam-bs [18:46] *** ndiddy has joined #archiveteam-bs [18:50] *** LFlare has joined #archiveteam-bs [19:23] *** Oddly has quit IRC (Ping timeout: 255 seconds) [19:55] *** Sk1d has quit IRC (Read error: Operation timed out) [19:57] *** Dimtree has quit IRC (Peace) [19:58] *** Sk1d has joined #archiveteam-bs [20:02] *** Dimtree has joined #archiveteam-bs [20:44] *** Exairnous has joined #archiveteam-bs [21:17] *** ndiddy has quit IRC (Ping timeout: 268 seconds) [21:33] *** Sk1d has quit IRC (Read error: Operation timed out) [21:36] *** Sk1d has joined #archiveteam-bs [21:48] *** BlueMax has joined #archiveteam-bs [21:54] *** Despatche has joined #archiveteam-bs [22:02] *** ndiddy has joined #archiveteam-bs [22:02] *** ndiddy has quit IRC (Client Quit) [22:49] *** schbirid has quit IRC (Remote host closed the connection) [22:50] *** omglolbah has joined #archiveteam-bs [23:12] jrwr: can we get support for svg on internetarchive.archiveteam.org? [23:13] *** wp494 has quit IRC (Ping timeout: 364 seconds) [23:14] *** wp494 has joined #archiveteam-bs [23:20] *** Stiletto has quit IRC (Read error: Operation timed out) [23:20] *** Stiletto has joined #archiveteam-bs