[00:01] WinterFox: 1 [00:01] ...sorry, WinterFox, that was a terrible way to typo /win [00:02] #klaraklok <- more spam there, less spam here [00:12] *** RichardG_ has joined #archiveteam [00:17] *** RichardG has quit IRC (Read error: Operation timed out) [00:25] *** Somebody has joined #archiveteam [00:26] *** VADemon has quit IRC (Quit: left4dead) [00:46] *** Aranje has joined #archiveteam [00:47] *** RichardG_ is now known as RichardG [01:17] *** Somebody has quit IRC (Ping timeout: 370 seconds) [01:20] *** maelstrom has joined #archiveteam [01:21] *** vOYtEC_ has joined #archiveteam [01:22] *** hawc145 has joined #archiveteam [01:24] *** wacky has quit IRC (Ping timeout: 250 seconds) [01:24] *** sHATNER has quit IRC (Ping timeout: 250 seconds) [01:24] *** Kksmkrn has quit IRC (Ping timeout: 250 seconds) [01:24] *** Deewiant has quit IRC (Ping timeout: 250 seconds) [01:24] *** C4K3 has quit IRC (Ping timeout: 250 seconds) [01:24] *** wacky has joined #archiveteam [01:24] *** C4K3 has joined #archiveteam [01:25] *** maelstrom has quit IRC (Ping timeout: 250 seconds) [01:25] *** HCross has quit IRC (Ping timeout: 250 seconds) [01:25] *** dashcloud has quit IRC (Ping timeout: 250 seconds) [01:25] *** dxdx has quit IRC (Ping timeout: 250 seconds) [01:25] *** vOYtEC has quit IRC (Ping timeout: 250 seconds) [01:25] *** pikhq has quit IRC (Ping timeout: 250 seconds) [01:25] *** TC02 has quit IRC (Ping timeout: 250 seconds) [01:25] *** Zebranky has quit IRC (Ping timeout: 250 seconds) [01:25] *** xhdr has quit IRC (Ping timeout: 250 seconds) [01:25] *** dashcloud has joined #archiveteam [01:25] *** Deewiant has joined #archiveteam [01:26] *** TC02 has joined #archiveteam [01:26] *** pikhq has joined #archiveteam [01:29] *** maelstrom has joined #archiveteam [01:29] *** Jonimus has quit IRC (Quit: WeeChat 1.4) [01:31] *** Jonimus has joined #archiveteam [01:31] *** Jonimus has quit IRC (Client Quit) [01:32] *** dx has joined #archiveteam [01:33] *** Zebranky has joined #archiveteam [01:49] *** Jonimus has joined #archiveteam [02:02] *** Jonimus has quit IRC (Quit: WeeChat 1.4) [02:03] *** Somebody has joined #archiveteam [02:08] *** Jonimus has joined #archiveteam [02:23] *** maelstrom has quit IRC (Ping timeout: 244 seconds) [02:33] *** maelstrom has joined #archiveteam [02:53] *** sHATNER has joined #archiveteam [02:54] *** xhdr has joined #archiveteam [02:54] *** xhdr has quit IRC (Excess Flood) [02:55] *** xhdr has joined #archiveteam [02:55] *** xhdr has quit IRC (Excess Flood) [02:57] *** xhdr has joined #archiveteam [02:57] *** xhdr has quit IRC (Excess Flood) [02:58] *** Kksmkrn has joined #archiveteam [02:58] *** xhdr has joined #archiveteam [02:58] *** xhdr has quit IRC (Excess Flood) [03:00] *** xhdr has joined #archiveteam [03:42] *** Lord_Nigh has quit IRC (Read error: Operation timed out) [03:44] *** ravetcofx has quit IRC (Read error: Operation timed out) [03:51] *** ravetcofx has joined #archiveteam [03:55] *** Lord_Nigh has joined #archiveteam [04:00] *** Lord_Nigh has quit IRC (Ping timeout: 244 seconds) [04:01] *** Lord_Nigh has joined #archiveteam [04:01] *** jrwr has quit IRC (Remote host closed the connection) [04:12] *** ndiddy has quit IRC (Read error: Connection reset by peer) [04:22] *** nicolas17 has quit IRC (Quit: zzz) [05:09] *** WinterFox has quit IRC (Remote host closed the connection) [05:09] *** Lord_Nigh has quit IRC (Read error: Operation timed out) [05:20] *** Lord_Nigh has joined #archiveteam [05:42] *** Sk1d has quit IRC (Ping timeout: 194 seconds) [05:48] *** Lord_Nigh has quit IRC (Ping timeout: 244 seconds) [05:48] *** Sk1d has joined #archiveteam [05:49] *** Lord_Nigh has joined #archiveteam [06:02] *** Aranje has quit IRC (Quit: Three sheets to the wind) [06:36] *** Lord_Nigh has quit IRC (Read error: Operation timed out) [06:38] *** Lord_Nigh has joined #archiveteam [06:46] *** Lord_Nigh has quit IRC (Read error: Operation timed out) [07:07] *** maelstrom has quit IRC (Quit: Leaving) [07:09] *** Lord_Nigh has joined #archiveteam [07:51] *** Lord_Nigh has quit IRC (Ping timeout: 250 seconds) [07:52] *** Lord_Nigh has joined #archiveteam [08:06] *** Somebody has quit IRC (Ping timeout: 370 seconds) [09:18] *** phuzion has quit IRC (Read error: Operation timed out) [09:19] *** phuzion has joined #archiveteam [09:22] *** ravetcofx has quit IRC (Read error: Operation timed out) [09:22] *** Lord_Nigh has quit IRC (Ping timeout: 244 seconds) [09:26] *** Lord_Nigh has joined #archiveteam [09:49] *** dashcloud has quit IRC (Ping timeout: 244 seconds) [09:51] *** dashcloud has joined #archiveteam [10:13] *** tomcat has joined #archiveteam [10:47] *** tomcat has quit IRC (Ping timeout: 194 seconds) [11:10] *** _Zialus_ has quit IRC (Read error: Operation timed out) [11:11] *** Zialus has joined #archiveteam [11:30] *** BlueMaxim has quit IRC (Quit: Leaving) [11:38] *** tomcat has joined #archiveteam [12:19] *** tomcat has quit IRC (Ping timeout: 194 seconds) [12:32] *** tomcat has joined #archiveteam [12:52] *** hawc145 is now known as HCross [13:07] *** BartoCH has quit IRC (Remote host closed the connection) [13:21] *** BartoCH has joined #archiveteam [13:52] *** tomcat has quit IRC (Remote host closed the connection) [13:58] *** qwebirc17 has joined #archiveteam [13:59] hey guys [13:59] are you going to archive pewdiepie's videos? [14:01] are they at risk? [14:02] http://kotaku.com/pewdiepie-swears-hell-delete-channel-at-50-million-subs-1789611322 [14:04] yeah ^ [14:05] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [14:07] i think we don't currently have a good method of archiving youtube channels with all metadata. [14:08] ive thrown it at youtube-dl on my server for now. 3k videos [14:08] solid [14:08] what about comments? [14:09] really not too sure what the best way to get that is. Unless we did grab-site over the videos [14:10] i'm pretty sure they require javascript. youtube comments have always been a mess [14:11] *** BartoCH has joined #archiveteam [14:13] 23/3018 [14:14] getting highest resolution, presuming [14:15] looks like i've broken youtube for myself by disabling scripts in noscript. now it won't load even after i disabled noscript lol [14:16] cool, thanks guys [14:16] youtube seems to be throttling my download speed [14:17] comments are horrible and full of requests to them are full of tokens [14:17] (excuse me) [14:19] https://github.com/philbot9/youtube-comment-scraper this is functional [14:20] *** qwebirc17 has quit IRC (Ping timeout: 268 seconds) [14:21] i'm going to see about downloading PewDiePie [14:21] Sanqui, no way of pointing it at a channel and saying "GET ALL COMMENTS" [14:22] true, but you can get a list of videos from youtube-dl [14:24] hm, how much disk is 3k videos at whatever quality he uploads at? [14:24] Im getting anywhere from 250MB to 1.2GB [15:00] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [15:52] *** kanzure has joined #archiveteam [15:52] hi, is there any way i could get an index of all archive.org torrent files, and/or all of the hashes of the .torrent files? [15:53] also, does anyone have that data, or know who i can pester to get that data? [15:53] *** zta_ has joined #archiveteam [15:53] *** zta_ has left [15:54] *** zta265 has joined #archiveteam [16:05] kanzure: The IA census data might contain info hashes. JesseW? [16:05] Example item: https://archive.org/details/ia-bak-census_20150304 [16:17] *** BartoCH has joined #archiveteam [16:24] *** kristian_ has joined #archiveteam [16:44] *** tomcat has joined #archiveteam [17:34] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [17:39] *** BartoCH has joined #archiveteam [17:45] *** BartoCH has quit IRC (Ping timeout: 260 seconds) [17:49] *** BartoCH has joined #archiveteam [18:08] *** zerkalo has quit IRC (Read error: Connection reset by peer) [18:08] *** zerkalo has joined #archiveteam [18:09] *** zerkalo has quit IRC (Read error: Connection reset by peer) [18:09] *** zerkalo has joined #archiveteam [18:10] *** ndiddy has joined #archiveteam [18:15] *** tomcat has quit IRC (Remote host closed the connection) [18:18] *** zerkalo has quit IRC (Ping timeout: 244 seconds) [18:20] internet archive has millions of torrents. [18:30] *** zerkalo has joined #archiveteam [18:40] *** maelstrom has joined #archiveteam [18:49] *** nicolas17 has joined #archiveteam [19:01] *** VADemon has joined #archiveteam [19:06] *** Somebody has joined #archiveteam [19:17] *** Somebody has quit IRC (Ping timeout: 370 seconds) [19:37] *** maelstrom has quit IRC (Quit: Leaving) [19:41] *** pfallenop has quit IRC (Remote host closed the connection) [19:46] *** ravetcofx has joined #archiveteam [19:52] *** BlueMaxim has joined #archiveteam [20:00] *** pfallenop has joined #archiveteam [20:06] *** Somebody has joined #archiveteam [20:33] *** maelstrom has joined #archiveteam [20:33] *** maelstrom has quit IRC (Client Quit) [21:09] *** jrwr has joined #archiveteam [21:21] *** Stiletto has quit IRC (Ping timeout: 244 seconds) [21:39] *** jsp234 has joined #archiveteam [21:41] *** jsp12345 has quit IRC (Read error: Operation timed out) [21:43] yes, i want all 32 million torrent hashes [21:44] PurpleSym: thank you. looks like they were using md5 O_o. [21:44] in 2015? [21:44] for passwords? [21:45] for hashes [21:45] of content [21:46] that's not really an issue is it? MD5 isn't cryptographically secure but that's irrelevant for uniqueness [21:46] making two files with the same MD5 hash isn't too hard nowadays [21:46] so yes it's relevant for uniqueness [21:47] i'm... hrm. not sure what to do about this. are they likely to regenerate hashes for all their 100s of petabytes of content? ouch. [21:47] by "isn't too hard" do you mean "likely to occur given a set of random data" or "one can artifically forge collisions" [21:47] in the former case it does matter, yes [21:47] Frogging: the latter [21:48] I have no context though :P [21:48] who should i poke over there to see about fixing that? [21:48] this is more of a #archiveteam-bs discussion, i think [21:49] got it, okay [21:49] also #internetarchive [21:49] er.. hi Sanqui. [21:49] this is now an archiveteam discussion but ia specific questions can go there [21:49] hi kanzure [21:50] nicolas17: he's talking about info hashes for torrent files, I think [21:50] nah, see -bs [21:50] *** jsp234 has quit IRC (Remote host closed the connection) [21:58] *** jsp12345 has joined #archiveteam [22:01] *** jsp12345 has quit IRC (Remote host closed the connection) [22:05] *** jsp12345 has joined #archiveteam [22:15] *** jsp12345 has quit IRC (Remote host closed the connection) [22:16] *** jsp12345 has joined #archiveteam [22:19] *** jsp12345 has quit IRC (Remote host closed the connection) [22:19] *** jsp12345 has joined #archiveteam [22:21] *** ndiddy has quit IRC (Read error: Connection reset by peer) [22:24] *** ndiddy has joined #archiveteam [22:30] *** jsp12345 has quit IRC (Remote host closed the connection) [22:30] *** jsp12345 has joined #archiveteam