[00:00] *** Sk1d has quit IRC (Read error: Operation timed out) [00:03] *** Sk1d has joined #archiveteam-bs [00:03] *** VerifiedJ has quit IRC (Quit: Leaving) [00:06] *** hiroi has joined #archiveteam-bs [00:54] *** BlueMax has joined #archiveteam-bs [01:30] *** alex____ has joined #archiveteam-bs [01:32] *** alex__ has quit IRC (Ping timeout: 265 seconds) [01:52] mandyfaq: I have an archiving thing that dumps YouTube videos to IA, but they only go to IA once gone from YouTube [01:52] if you need them on IA regardless of YouTube status I have no opinion on what to do [01:57] *** Ryz has quit IRC (Remote host closed the connection) [02:06] latest scan : https://archive.org/details/pc-computing-magazine-v7i5 [02:24] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [02:35] *** TigerbotH has quit IRC (ZNC - http://znc.in) [02:40] *** TigerbotH has joined #archiveteam-bs [03:01] *** Tenebrae has quit IRC (Read error: Operation timed out) [03:02] *** Tenebrae has joined #archiveteam-bs [03:16] @ivan My current model was to scrape links from a wiki site RSS feed and immediately use TubeUp to back them up to IA. I suppose I could just download them and wait until they go down before uploading to IA, but I don't see any benefit. [03:18] plus it means having to add a HDD onto the Pi i was planning to run it on, so if there isn't a good reason I'd rather not [03:18] btw what is your archiving thing targetting? general YouTube or ceratin categories? [03:19] *certain [03:19] Whatever people ask him and whatever he thinks might go down [03:22] nice [03:31] i think i wont add my bot's uploads to mirrortube collection, to avoid confusion. [03:31] also big thanks to all the ArchiveTeam projects. great work! [03:32] mandyfaq send the list to ivan [03:33] mandyfaq send the list to ivan [03:33] well its not just a list [03:33] ive made a scraper so when new links are posted they get saved [03:34] and i add more stuff to the description so that people can find on what page it was linked too from [03:35] *** Sk1d has quit IRC (Read error: Operation timed out) [03:35] and there are expected to be many dead links which get categorised [03:38] *** Sk1d has joined #archiveteam-bs [03:55] *** decay has quit IRC (Ping timeout: 252 seconds) [03:55] *** decay has joined #archiveteam-bs [04:02] *** Kitaru has joined #archiveteam-bs [04:20] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [04:24] *** hiroi has quit IRC (Read error: Operation timed out) [04:25] *** hiroi has joined #archiveteam-bs [04:29] *** qw3rty118 has joined #archiveteam-bs [04:30] *** odemgi has joined #archiveteam-bs [04:30] *** archi__ has joined #archiveteam-bs [04:31] *** odemgi_ has quit IRC (Read error: Operation timed out) [04:32] *** qw3rty117 has quit IRC (Ping timeout: 600 seconds) [04:33] *** archi_ has quit IRC (Ping timeout: 252 seconds) [04:33] *** odemg has quit IRC (Ping timeout: 265 seconds) [04:37] *** Kitaru has joined #archiveteam-bs [04:43] *** Sk1d has quit IRC (Read error: Operation timed out) [04:45] *** odemg has joined #archiveteam-bs [04:47] *** Sk1d has joined #archiveteam-bs [04:49] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [04:56] *** Martle has quit IRC (Remote host closed the connection) [04:59] *** Sk1d has quit IRC (Read error: Operation timed out) [05:04] *** Sk1d has joined #archiveteam-bs [05:15] *** Kitaru has joined #archiveteam-bs [05:16] *** Sk1d has quit IRC (Read error: Operation timed out) [05:21] *** Sk1d has joined #archiveteam-bs [05:33] *** godane has quit IRC (Ping timeout: 252 seconds) [05:34] *** godane has joined #archiveteam-bs [05:35] *** Sk1d has quit IRC (Read error: Operation timed out) [05:39] *** fredgido has quit IRC (Remote host closed the connection) [05:39] *** SimpBrain has quit IRC (Read error: Connection reset by peer) [05:39] *** fredgido has joined #archiveteam-bs [05:41] *** Sk1d has joined #archiveteam-bs [05:41] *** SimpBrain has joined #archiveteam-bs [06:22] *** mandyfaq has quit IRC (Quit: Page closed) [06:37] *** Ryz has joined #archiveteam-bs [06:58] *** hdch has quit IRC (Quit: Leaving) [07:21] *** Sk1d has quit IRC (Read error: Operation timed out) [07:26] *** Sk1d has joined #archiveteam-bs [07:29] *** hdch has joined #archiveteam-bs [07:34] *** alex____ has quit IRC (Quit: alex____) [08:27] *** Kitaru has quit IRC (Quit: This computer has gone to sleep) [08:57] *** alex___ has joined #archiveteam-bs [09:02] *** alex___ has quit IRC (Quit: alex___) [09:04] *** alex___ has joined #archiveteam-bs [09:07] *** Sk1d has quit IRC (Read error: Operation timed out) [09:09] *** Sk1d has joined #archiveteam-bs [09:15] *** BlueMax has quit IRC (Read error: Connection reset by peer) [09:21] *** Sk1d has quit IRC (Read error: Operation timed out) [09:26] *** Sk1d has joined #archiveteam-bs [09:30] *** zyphlar has joined #archiveteam-bs [09:30] PurpleSym: you joined it on matrix and it didn't load? I'm talking to you through it... [09:31] *** zyphlar has quit IRC (Remote host closed the connection!) [09:33] *** Ing3b0rg has quit IRC (Read error: Operation timed out) [09:39] *** TC01 has quit IRC (Read error: Operation timed out) [09:43] *** hdch has quit IRC (Remote host closed the connection) [10:15] *** Smiley has quit IRC (Read error: Operation timed out) [10:16] *** Smiley has joined #archiveteam-bs [11:05] *** archi__ has quit IRC (Remote host closed the connection) [11:28] *** Ryz has quit IRC (Quit: ChatZilla 0.9.92-rdmsoft [XULRunner 35.0.1/20150122214805]) [12:40] *** fredgido has quit IRC (Read error: Connection reset by peer) [12:41] *** fredgido has joined #archiveteam-bs [12:53] *** Sk1d has quit IRC (Read error: Operation timed out) [12:56] *** Sk1d has joined #archiveteam-bs [13:30] *** Martle has joined #archiveteam-bs [14:51] *** Sk1d has quit IRC (Read error: Operation timed out) [14:54] *** Sk1d has joined #archiveteam-bs [15:17] *** icedice has quit IRC (Leaving) [15:36] *** TC01 has joined #archiveteam-bs [15:40] *** icedice has joined #archiveteam-bs [15:54] *** marked has joined #archiveteam-bs [17:10] *** Sk1d has quit IRC (Read error: Operation timed out) [17:14] *** Sk1d has joined #archiveteam-bs [17:24] *** schbirid has joined #archiveteam-bs [17:35] Actually, nevermind the ping. So regarding static.xx.fbcdn.net, Facebook's static CDN: each Facebook page links to thousands of files on that CDN, although most of them are probably not even used. That's what Facebook does; they're running on a single 1+ GB PHP executable after all (or were doing so a few years ago). [17:35] What this means is that playback of Facebook pages *might* be broken if you skip those links. [17:36] *** VerifiedJ has joined #archiveteam-bs [17:36] If you use wpull for crawling, it will also extract a lot of extra URLs from within the JS files hosted on that domain, and a good number of those will be invalid URLs which just get a status code 400. It's safe to ignore those when wpull retries them. [17:37] sec^nd: ^ [17:37] The JS is also pulled in whenever anything from Facebook appears on a site, e.g. a like button. [17:42] *** Martle has quit IRC (Quit: Leaving) [17:45] We should make a channel for Gab, any ideas? [17:48] #shutup [17:55] gape? [18:13] *** hdch has joined #archiveteam-bs [18:19] *** Ryz has joined #archiveteam-bs [18:53] *** Mateon1 has quit IRC (Ping timeout: 252 seconds) [18:53] *** Mateon1 has joined #archiveteam-bs [19:07] *** xarph_ is now known as xarph [19:11] *** Sk1d has quit IRC (Read error: Operation timed out) [19:11] *** icedice has quit IRC (Leaving) [19:16] *** Sk1d has joined #archiveteam-bs [19:44] so i scanned 416 pages today [19:44] 1994-06 issue of pc computing was very big [20:04] so the dtic.mil is down [20:10] *** Stiletto has quit IRC () [20:26] *** Kitaru has joined #archiveteam-bs [20:46] *** Stiletto has joined #archiveteam-bs [20:55] *** Sk1d has quit IRC (Read error: Operation timed out) [21:01] *** Sk1d has joined #archiveteam-bs [21:02] *** BlueMax has joined #archiveteam-bs [21:33] *** schbirid has quit IRC (Remote host closed the connection) [22:33] *** alex___ has quit IRC (Quit: ZZzzz) [23:14] *** hdch has quit IRC (Quit: Leaving) [23:43] *** hdch has joined #archiveteam-bs [23:49] *** Verified_ has joined #archiveteam-bs [23:50] *** VerifiedJ has quit IRC (Ping timeout: 252 seconds) [23:53] *** marked has quit IRC (Remote host closed the connection) [23:57] *** Jens has quit IRC (Remote host closed the connection) [23:58] *** Jens has joined #archiveteam-bs [23:58] *** Verified_ is now known as VerifiedJ