[00:08] *** MMovie has quit IRC (Read error: Operation timed out) [00:08] *** MMovie has joined #archiveteam [00:09] MrRadar: can you grab it all then for us please? [00:10] My upload isn't very fast but these films are only in 480p quality so I probably could. [00:11] How should I be grabbing them? youtube-dl, grab-site? [00:11] YouTube dl [00:11] *** Stiletto has joined #archiveteam [00:11] Plain files or WARCs? [00:11] Plain files [00:12] MP4 preferred, I'd assume? [00:13] I'll start it when I get back home. Where should these end up eventually? [00:13] The IA [00:14] By "eventually" I meant after I grab them, where should I put them? Directly to the IA? [00:14] If you can [00:15] If I uploaded them to the IA under my account, would they ban me for uploading so much copyrighted material? [00:16] It would just get darkened I think [00:16] SketchCow (or anyone who knows the IA's policies): can you confirm that? [00:17] I'm going AFK for about an hour now [00:37] *** MMovie has quit IRC (Read error: Operation timed out) [00:38] *** MMovie has joined #archiveteam [00:52] OK I really need to change this debian instance to run level 3 instead of 5 but I can't find how to do that anywhere, inittab doesn't exisit /etc/rc.d/rc-sysint.conf /etc/systemd/system/default.target doesn't exist [00:52] like none of the things exist [00:52] sounds like you got converted to systemd [00:52] you're on your own now bub [00:53] also this is offtopic, take it to #archiveteam-bs or somewhere else [00:53] *** MMovie has quit IRC (Read error: Operation timed out) [00:53] ugh I found it, and I'm traking to make an archiveteam host work sooooooooooooooooooooooooooo there's that [00:53] *** Ungstein has joined #archiveteam [00:54] *** MMovie has joined #archiveteam [01:08] *** MMovie has quit IRC (Read error: Operation timed out) [01:10] *** MMovie has joined #archiveteam [01:19] *** Stiletto has quit IRC () [01:20] *** nertzy has quit IRC (Read error: Operation timed out) [01:25] *** nertzy has joined #archiveteam [01:29] *** acridAxid has quit IRC (Quit: marauder) [01:30] *** acridAxid has joined #archiveteam [01:35] *** plog99 has quit IRC (Read error: Connection reset by peer) [01:37] *** plog99 has joined #archiveteam [01:39] *** Stiletto has joined #archiveteam [01:44] *** bsmith093 has joined #archiveteam [01:47] *** philpem has quit IRC (Ping timeout: 260 seconds) [01:54] *** MMovie has quit IRC (Read error: Operation timed out) [01:55] *** MMovie has joined #archiveteam [02:13] *** MMovie has quit IRC (Read error: Operation timed out) [02:13] *** MMovie has joined #archiveteam [02:23] *** vitzli has joined #archiveteam [02:26] *** JesseW has joined #archiveteam [02:27] *** MMovie has quit IRC (Read error: Operation timed out) [02:28] *** MMovie has joined #archiveteam [02:30] *** Stiletto has quit IRC (Ping timeout: 258 seconds) [02:34] *** DopefishJ is now known as DFJustin [02:44] *** MMovie has quit IRC (Read error: Operation timed out) [02:45] *** MMovie has joined #archiveteam [02:50] *** tomwsmf-a has joined #archiveteam [02:50] Oh look, i woke up [02:50] *** Atom__ has joined #archiveteam [02:51] *** mr-b has quit IRC (Read error: Operation timed out) [02:51] If you upload a bunch of crazy shit, they will go to me "Is this one of yours?" [02:51] And I will hide you in my petticoat [02:51] *** aMunster has quit IRC (Read error: Operation timed out) [02:52] *** mr-b has joined #archiveteam [02:52] *** rossdylan has quit IRC (Read error: Operation timed out) [02:52] *** lbft_ has quit IRC (Read error: Operation timed out) [02:53] *** closure has quit IRC (Read error: Operation timed out) [02:53] *** remsen has quit IRC (Read error: Operation timed out) [02:53] *** remsen has joined #archiveteam [02:53] *** lbft has joined #archiveteam [02:54] *** Atom has quit IRC (Read error: Operation timed out) [02:55] *** Jonimus has quit IRC (Read error: Operation timed out) [02:55] *** phuzion has quit IRC (Read error: Operation timed out) [02:57] *** phuzion has joined #archiveteam [02:58] *** aMunster has joined #archiveteam [02:59] *** acridAxid has quit IRC (Read error: Operation timed out) [02:59] *** acridAxid has joined #archiveteam [02:59] *** antomati_ has joined #archiveteam [02:59] *** swebb sets mode: +o antomati_ [03:00] *** Jonimus has joined #archiveteam [03:00] *** swebb sets mode: +o Jonimus [03:01] *** winr5r has quit IRC (Ping timeout: 634 seconds) [03:01] *** mhazinsk has quit IRC (Ping timeout: 634 seconds) [03:01] *** winr4r has joined #archiveteam [03:02] *** nwf has quit IRC (Ping timeout: 633 seconds) [03:02] *** aMunster has quit IRC (Remote host closed the connection) [03:03] *** lysobit has quit IRC (Read error: Operation timed out) [03:03] *** musalbas has joined #archiveteam [03:03] *** musalbas is now known as lysobit [03:04] *** Swizzle__ has quit IRC (Quit: Leaving) [03:05] *** w0rp has quit IRC (Ping timeout: 633 seconds) [03:05] *** nwf has joined #archiveteam [03:06] *** Mayonaise has quit IRC (Ping timeout: 864 seconds) [03:08] *** w0rp has joined #archiveteam [03:10] *** Stiletto has joined #archiveteam [03:11] *** antomatic has quit IRC (Ping timeout: 865 seconds) [03:13] *** marvinw has quit IRC (Write error: Broken pipe) [03:13] *** marvinw_ has joined #archiveteam [03:16] *** vegbrasil has quit IRC (Ping timeout: 633 seconds) [03:19] *** MMovie has quit IRC (Read error: Operation timed out) [03:19] *** vegbrasil has joined #archiveteam [03:19] *** Mayonaise has joined #archiveteam [03:20] *** aMunster has joined #archiveteam [03:20] for youtube-dl, if I'm downloading a whole series of videos from a channel, and it gets stuck or fails before the end, I just need to use -w to keep the existing files in place, and then the rest of the files in the list will be retrieved right? [03:20] *** MMovie has joined #archiveteam [03:22] *** closure has joined #archiveteam [03:22] *** mhazinsk has joined #archiveteam [03:30] *** vitzli has quit IRC (Leaving) [03:33] *** acridAxid has quit IRC (Read error: Operation timed out) [03:33] *** mr-b has quit IRC (Read error: Operation timed out) [03:34] *** SadDM has quit IRC (Read error: Operation timed out) [03:34] *** jspiros has quit IRC (Read error: Operation timed out) [03:34] *** brabo has quit IRC (Read error: Operation timed out) [03:35] *** JesseW has quit IRC (Quit: Leaving.) [03:35] *** Atom__ has quit IRC (Read error: Operation timed out) [03:35] *** matthusb- has quit IRC (Ping timeout: 246 seconds) [03:35] *** remsen has quit IRC (Ping timeout: 246 seconds) [03:37] *** JesseW has joined #archiveteam [03:39] *** wp494 has quit IRC (Ping timeout: 362 seconds) [03:40] *** remsen has joined #archiveteam [03:41] *** acridAxid has joined #archiveteam [03:42] *** JesseW has quit IRC (Client Quit) [03:46] *** mr-b has joined #archiveteam [03:55] *** MMovie has quit IRC (Read error: Operation timed out) [03:56] *** MMovie has joined #archiveteam [04:04] so, to answer this for anyone wondering- that's what you need to do to not have the files overwritten' [04:12] Internet Archive is having a host of issues/problems, so it keeps going up and down. Ironically it's because we're making search and other facilities more robust. [04:12] *** MMovie has quit IRC (Read error: Operation timed out) [04:13] *** MMovie has joined #archiveteam [04:26] *** bwn has quit IRC (Read error: Operation timed out) [04:33] dashcloud: if it's done with a video, redownloading the whole list will skip that one, use -i to ignore errors [04:39] *** tomwsmf-a has quit IRC (Ping timeout: 258 seconds) [04:43] *** MMovie has quit IRC (Read error: Operation timed out) [04:45] *** MMovie has joined #archiveteam [04:56] *** Stiletto has quit IRC (Read error: Operation timed out) [04:57] *** Stiletto has joined #archiveteam [05:17] *** Sk1d has quit IRC (Ping timeout: 250 seconds) [05:24] *** Sk1d has joined #archiveteam [05:36] *** MMovie has quit IRC (Read error: Operation timed out) [05:37] *** MMovie has joined #archiveteam [05:48] *** bwn has joined #archiveteam [05:51] *** Atom has joined #archiveteam [05:55] *** kcaj has quit IRC (Ping timeout: 258 seconds) [05:55] *** kcaj has joined #archiveteam [06:02] *** WinterFox has joined #archiveteam [06:04] *** nwf has quit IRC (Read error: Operation timed out) [06:05] *** JesseW has joined #archiveteam [06:05] *** mr-b has quit IRC (Read error: Operation timed out) [06:06] *** aMunster has quit IRC (Read error: Operation timed out) [06:06] *** vegbrasil has quit IRC (Read error: Operation timed out) [06:06] *** mhazinsk has quit IRC (Read error: Operation timed out) [06:07] *** beardicus has quit IRC (Read error: Operation timed out) [06:08] *** remsen has quit IRC (Ping timeout: 364 seconds) [06:08] *** Mayonaise has quit IRC (Read error: Operation timed out) [06:10] *** Jonimus has quit IRC (Read error: Operation timed out) [06:10] *** Mayonaise has joined #archiveteam [06:18] *** closure has quit IRC (Ping timeout: 633 seconds) [06:24] *** MMovie has quit IRC (Read error: Operation timed out) [06:25] *** MMovie has joined #archiveteam [06:33] *** vegbrasil has joined #archiveteam [06:33] *** closure has joined #archiveteam [06:33] *** remsen has joined #archiveteam [06:36] *** aMunster has joined #archiveteam [06:39] *** Sk2d has joined #archiveteam [06:41] *** rossdylan has joined #archiveteam [06:43] *** mhazinsk has joined #archiveteam [06:43] *** mr-b has joined #archiveteam [06:45] *** Sk1d has quit IRC (hub.se irc.du.se) [06:47] *** MMovie has quit IRC (Read error: Operation timed out) [06:48] *** MMovie has joined #archiveteam [06:49] *** Jonimus has joined #archiveteam [06:49] *** swebb sets mode: +o Jonimus [06:53] *** nwf has joined #archiveteam [07:00] *** Sk2d is now known as Sk1d [07:01] *** beardicus has joined #archiveteam [07:03] *** JesseW has quit IRC (Quit: Leaving.) [07:08] *** MMovie has quit IRC (Read error: Operation timed out) [07:09] *** MMovie has joined #archiveteam [07:18] *** godane has quit IRC (Quit: Leaving.) [07:20] *** godane has joined #archiveteam [07:21] *** jut has joined #archiveteam [07:43] *** MMovie has quit IRC (Read error: Operation timed out) [07:44] *** MMovie has joined #archiveteam [08:01] *** MMovie has quit IRC (Read error: Operation timed out) [08:02] *** MMovie has joined #archiveteam [08:11] *** atomotic has joined #archiveteam [08:19] *** schbirid has joined #archiveteam [08:30] *** MMovie has quit IRC (Read error: Operation timed out) [08:31] *** MMovie has joined #archiveteam [08:37] SketchCow: http://i.imgur.com/l93g1cv.gif [08:57] *** MMovie has quit IRC (Read error: Operation timed out) [08:59] *** MMovie has joined #archiveteam [09:12] *** MMovie has quit IRC (Read error: Operation timed out) [09:14] *** MMovie has joined #archiveteam [09:34] *** bwn has quit IRC (Read error: Operation timed out) [09:46] *** bwn has joined #archiveteam [09:54] *** MMovie has quit IRC (Read error: Operation timed out) [09:55] *** MMovie has joined #archiveteam [10:05] *** vOYtEC has joined #archiveteam [10:11] *** jut has quit IRC (Read error: Connection reset by peer) [10:11] *** MMovie has quit IRC (Read error: Operation timed out) [10:12] *** MMovie has joined #archiveteam [10:17] *** jut has joined #archiveteam [10:27] *** MMovie has quit IRC (Read error: Operation timed out) [10:28] *** MMovie has joined #archiveteam [10:33] Does someone have a rsync target available with a few GB or space? [10:34] We think fotolog might speed up a lot if we use another rsync target for the discovery part of fotolog [10:35] First need to test that idea before we make permanent changes, but that's why we'd like to test with a diferent rsync target [10:35] So if you have something available, please let me know!! [10:39] how much you need? [10:43] arkiver: dat.serveert.me.uk::fotolog should be up, has about 200GB free atm [10:47] tested, should work [10:50] *** MMovie has quit IRC (Read error: Operation timed out) [10:52] *** MMovie has joined #archiveteam [11:02] I could set up 1 TB as well if needed. [11:06] midas: thanks, you and Fletcher both have a target ready [11:06] * arkiver is afk for 4 hours [11:06] We'll add a target after that [11:07] *** MMovie has quit IRC (Read error: Operation timed out) [11:08] *** MMovie has joined #archiveteam [11:10] I think we'll use the target with the best IO and CPU [11:10] Since we need to deduplicate large lists, which might take some processing power [11:12] midas: Fletcher: how is the CPU on your box? [11:13] arkiver: if you want to send some stuff over to that target I sent you last night, feel free. If you want, I can set another target up for you [11:15] arkiver: not that fast [11:16] CPU: 4 x Intel Atom (1862.000 MHz) [11:16] Fletcher: if you think your box has more processing power we'll go with that rsync [11:16] midas: thanks [11:16] * arkiver is afk for 4 hours [11:19] *** fpoee has joined #archiveteam [11:23] *** plog99 has quit IRC (Read error: Operation timed out) [11:27] *** vitzli has joined #archiveteam [11:30] *** davidar has quit IRC (Quit: Connection closed for inactivity) [11:45] *** MMovie has quit IRC (Read error: Operation timed out) [11:47] *** MMovie has joined #archiveteam [11:57] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [12:02] *** Froggypwn has quit IRC (Quit: ~ Trillian Astra - www.trillian.im ~) [12:17] *** MMovie has quit IRC (Read error: Operation timed out) [12:18] *** MMovie has joined #archiveteam [12:36] *** MMovie has quit IRC (Read error: Operation timed out) [12:37] *** MMovie has joined #archiveteam [12:41] *** atomotic has joined #archiveteam [12:49] *** matthusb- has joined #archiveteam [12:50] *** SadDM has joined #archiveteam [12:50] *** swebb sets mode: +o SadDM [12:51] *** WinterFox has quit IRC (Remote host closed the connection) [12:51] *** brabo has joined #archiveteam [12:53] *** jspiros has joined #archiveteam [13:01] *** toad2 has joined #archiveteam [13:02] *** toad1 has quit IRC (Read error: Operation timed out) [13:22] *** MMovie has quit IRC (Read error: Operation timed out) [13:23] *** MMovie has joined #archiveteam [13:25] *** atomotic has quit IRC (Quit: My Mac has gone to sleep. ZZZzzz…) [13:39] *** MMovie has quit IRC (Read error: Operation timed out) [13:40] *** MMovie has joined #archiveteam [13:50] *** ersi has quit IRC (Read error: Operation timed out) [13:56] *** ersi has joined #archiveteam [13:56] *** swebb sets mode: +o ersi [14:15] *** MMovie has quit IRC (Read error: Operation timed out) [14:17] *** MMovie has joined #archiveteam [14:30] *** MMovie has quit IRC (Read error: Operation timed out) [14:32] *** MMovie has joined #archiveteam [14:48] *** atomotic has joined #archiveteam [14:48] *** MMovie has quit IRC (Read error: Operation timed out) [14:49] *** MMovie has joined #archiveteam [15:07] *** MMovie has quit IRC (Read error: Operation timed out) [15:08] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [15:08] *** MMovie has joined #archiveteam [15:09] *** tomwsmf-a has joined #archiveteam [15:17] *** Start has quit IRC (Quit: Disconnected.) [15:23] *** MMovie has quit IRC (Read error: Operation timed out) [15:25] *** MMovie has joined #archiveteam [15:26] *** RichardG has quit IRC (Ping timeout: 258 seconds) [15:36] *** Froggypwn has joined #archiveteam [15:39] *** RichardG has joined #archiveteam [15:42] *** philpem has joined #archiveteam [15:48] *** Start has joined #archiveteam [15:54] *** MMovie has quit IRC (Read error: Operation timed out) [15:55] *** MMovie has joined #archiveteam [16:07] We have an insider on BetaArchive. He offers to free it, and needs a place to put what he gets. Who's got a lot (a LOT) of space, so we can ingest it and then put it into the Archive? [16:09] How much is a lot? [16:09] error, a lot = undef [16:12] *** lysobit has quit IRC (Ping timeout: 260 seconds) [16:14] 22tb, ultimately, although we'd be doing it in stages [16:15] I have asked for a file listing because I want to understand what exactly this is. [16:20] Maybe Kenshin or zino_ have some space available [16:21] Though zino_ is working with us on the FTP project and hold gametrailers, so cino_ might not have the space for it currently [16:25] *** lysobit has joined #archiveteam [16:27] *** JesseW has joined #archiveteam [16:32] *** MMovie has quit IRC (Read error: Operation timed out) [16:33] *** MMovie has joined #archiveteam [16:41] *** scyther has joined #archiveteam [16:48] *** MMovie has quit IRC (Read error: Operation timed out) [16:49] *** JesseW has quit IRC (Quit: Leaving.) [16:49] *** MMovie has joined #archiveteam [16:51] *** Zei-Pii has joined #archiveteam [17:01] *** tomwsmf-a has quit IRC (Read error: Operation timed out) [17:04] *** Simpbrai1 has quit IRC (Ping timeout: 260 seconds) [17:07] *** Start has quit IRC (Quit: Disconnected.) [17:14] *** VADemon has joined #archiveteam [17:17] *** Simpbrai_ has joined #archiveteam [17:22] *** tomwsmf-a has joined #archiveteam [17:23] *** MMovie has quit IRC (Read error: Operation timed out) [17:25] *** MMovie has joined #archiveteam [17:39] *** VADemon has quit IRC (Read error: Connection reset by peer) [17:41] With 8.6T used for GT and 5T mentally allocated for FTP I only have slightly above 12T free on the current machine. [17:44] I could hook up a JBOD with another 20T, but that's not going to happen before Easter. [17:45] *** atomotic has joined #archiveteam [17:47] Well, I seriously doubt this guy is going to upload more than 1-2tv [17:47] tb [17:47] *** pgoetz has joined #archiveteam [17:49] I can probably take it, if its just 1-2TB [18:01] SketchCow: Sent you a PM in case you decide you need to use my machine. [18:04] *** MMovie has quit IRC (Read error: Operation timed out) [18:06] *** MMovie has joined #archiveteam [18:08] I want help setting up my machine to be ready as a rsync target [18:08] then we can maybe test the speed of it [18:08] see if it is usfull as anything more than a archivebot-node [18:09] the best would be if we could documents what is needed in the proccess, so others have some place to go in the future [18:10] *** atomotic has quit IRC (Quit: Textual IRC Client: www.textualapp.com) [18:12] *** scyther has quit IRC (Quit: Leaving) [18:12] *** db48x has joined #archiveteam [18:19] I imagine the list would be pretty short: [18:19] 1. Have bandwidth. [18:19] 2. Have a computer somewhere where it won't be fiddled with. [18:19] 3. Have a bit of disk that is raided and monitored so that all the [18:19] collected data doesn't go *poof* [18:20] Bonus points: [18:20] 1. Know how to properly set up rsyncd. [18:21] *** vitzli has quit IRC (Quit: Leaving) [18:28] *** Tomcat_ has joined #archiveteam [18:33] *** Tomcat_ has quit IRC (Ping timeout: 260 seconds) [18:33] *** MMovie has quit IRC (Read error: Operation timed out) [18:35] *** MMovie has joined #archiveteam [18:48] *** Tomcat_ has joined #archiveteam [18:52] *** MMovie has quit IRC (Read error: Operation timed out) [18:54] *** MMovie has joined #archiveteam [19:04] *** MMovie has quit IRC (Read error: Operation timed out) [19:05] *** MMovie has joined #archiveteam [19:16] *** MMovie has quit IRC (Read error: Operation timed out) [19:18] *** MMovie has joined #archiveteam [19:28] *** Start has joined #archiveteam [19:48] *** MMovie has quit IRC (Read error: Operation timed out) [19:50] *** MMovie has joined #archiveteam [19:54] *** tigerteam has joined #archiveteam [19:54] *** tigerteam has quit IRC (Client Quit) [19:54] *** tigris37 has joined #archiveteam [19:54] *** tigris37 has quit IRC (Client Quit) [20:01] *** JW_work1 has joined #archiveteam [20:03] *** JW_work has quit IRC (Ping timeout: 362 seconds) [20:09] *** lkmlk has joined #archiveteam [20:09] *** lkmlk has quit IRC (Connection closed) [20:29] *** MMovie has quit IRC (Read error: Operation timed out) [20:29] *** bwn has quit IRC (Read error: Operation timed out) [20:31] *** MMovie has joined #archiveteam [20:34] *** WinterFox has joined #archiveteam [20:35] *** Ghost_of_ has joined #archiveteam [20:46] *** Start has quit IRC (Quit: Disconnected.) [20:48] *** bwn has joined #archiveteam [20:50] *** MMovie has quit IRC (Read error: Operation timed out) [20:51] *** MMovie has joined #archiveteam [20:58] *** Start has joined #archiveteam [20:59] *** JW_work has joined #archiveteam [21:02] *** wp494 has joined #archiveteam [21:02] *** JW_work1 has quit IRC (Ping timeout: 362 seconds) [21:20] *** MMovie has quit IRC (Read error: Operation timed out) [21:22] *** MMovie has joined #archiveteam [21:40] *** MMovie has quit IRC (Read error: Operation timed out) [21:42] *** MMovie has joined #archiveteam [21:45] *** jut has quit IRC (Quit: jut) [21:59] *** MMovie has quit IRC (Read error: Operation timed out) [22:01] *** MMovie has joined #archiveteam [22:05] *** Ghost_of_ has quit IRC (Quit: Leaving) [22:13] *** schbirid has quit IRC (Quit: Leaving) [22:16] *** Start has quit IRC (Quit: Disconnected.) [22:23] *** MMovie has quit IRC (Read error: Operation timed out) [22:24] *** VADemon has joined #archiveteam [22:24] *** MMovie has joined #archiveteam [22:42] *** MMovie has quit IRC (Read error: Operation timed out) [22:44] *** MMovie has joined #archiveteam [22:47] *** Tomcat_ has quit IRC (Remote host closed the connection) [23:00] *** MMovie has quit IRC (Read error: Operation timed out) [23:01] *** MMovie has joined #archiveteam [23:17] *** Start has joined #archiveteam [23:22] *** FalconK has quit IRC (Ping timeout: 260 seconds) [23:36] *** MMovie has quit IRC (Read error: Operation timed out) [23:37] *** MMovie has joined #archiveteam [23:43] *** philpem has quit IRC (Ping timeout: 260 seconds)