[00:01] letproject newsletter discussion [00:01] arkiver: probably not [00:02] let's talk project newsletter on #projectnewsletter [00:02] accidentally hit enter instead of backspace -_- [00:02] *** mistym has quit IRC (Remote host closed the connection) [00:03] chfoo: we put it in archivebot, what was discovered [00:03] * arkiver is afk for the night, good day/night all!! [00:08] *** dashcloud has quit IRC (Read error: Operation timed out) [00:18] *** dashcloud has joined #archiveteam [00:52] "from march 29, this device no longer supports Music Unlimited" [00:52] fuck off [00:52] my device supports it just fine [00:52] YOU don't support it [00:52] *** garyrh has joined #archiveteam [00:54] basically. [00:55] *** Ymgve has quit IRC () [01:01] *** mistym has joined #archiveteam [01:38] *** mistym has quit IRC (Remote host closed the connection) [01:39] *** mistym has joined #archiveteam [01:57] *** mistym_ has joined #archiveteam [02:02] *** mistym has quit IRC (Read error: Operation timed out) [02:10] *** VADemon has quit IRC (Read error: Connection reset by peer) [02:16] *** Stiletto has joined #archiveteam [03:24] *** primus104 has quit IRC (Leaving.) [03:38] *** mistym_ has quit IRC (Remote host closed the connection) [03:39] *** SN4T14_ has quit IRC (Read error: Connection reset by peer) [03:39] *** Emcy has quit IRC (Read error: Connection reset by peer) [03:39] *** Emcy has joined #archiveteam [03:39] *** SN4T14_ has joined #archiveteam [03:42] *** SN4T14_ has quit IRC (Read error: Connection reset by peer) [03:42] *** SN4T14_ has joined #archiveteam [04:04] *** mistym has joined #archiveteam [04:05] *** aaaaaaaaa has quit IRC (Leaving) [04:26] *** Nertsy has quit IRC (Quit: Nertsy) [04:29] *** Nertsy has joined #archiveteam [05:14] *** cadbury_ has quit IRC (Read error: Connection reset by peer) [05:15] *** aNthraXx_ has joined #archiveteam [05:15] *** aNthraXx has quit IRC (Read error: Connection reset by peer) [05:15] *** cadbury_ has joined #archiveteam [05:24] *** JMC_ is now known as JMC [06:42] *** redyoshi4 has joined #archiveteam [06:43] hey, I'd like to make an account on archiveteam.org's wiki so as to add a site to the deathwatch list, and I need the secret word for account creation [06:44] WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD [06:44] (in case the exact text is necessary for that) [06:45] yahoosucks [06:45] that's the secred word? [06:45] yep [06:45] *secret [06:45] OK [06:46] *** qwebirc77 has joined #archiveteam [06:46] bah; net connection dropped on me [06:46] thanks [06:47] (I was redyoshi4) [06:50] *** redyoshi4 has quit IRC (Ping timeout: 240 seconds) [07:08] *** qwebirc77 is now known as redyoshi4 [07:09] *** mistym has quit IRC (Remote host closed the connection) [07:17] *** mistym has joined #archiveteam [07:25] *** SimpBrain has joined #archiveteam [07:28] *** mistym has quit IRC (Read error: Operation timed out) [07:29] http://archiveteam.org/index.php?title=Deathwatch#FurAffinity <- I finished making my entry; to double check, is it in the right place? [08:24] *** dashcloud has quit IRC (Read error: Operation timed out) [08:30] *** dashcloud has joined #archiveteam [08:36] *** schbirid has joined #archiveteam [09:35] *** Ymgve has joined #archiveteam [10:23] *** signius has quit IRC (Read error: Operation timed out) [10:30] *** BlueMaxim has quit IRC (Quit: Leaving) [10:33] *** habi has joined #archiveteam [10:35] *** signius has joined #archiveteam [10:37] *** Mayonaise has quit IRC (Ping timeout: 512 seconds) [10:41] *** Mayonaise has joined #archiveteam [10:47] *** habi has left [10:56] this is a rather nice command.. parallel unrar t ::: ./**/*.rar | grep -v OK [10:56] wonder if there's a nicer way to get clear feedback on exactly which archives failed the test though [10:56] there must be some combination of flags that would make things clearer [11:39] johtso: maybe you want the -L flag from the grep man page? [11:40] "Suppress normal output; instead print the name of each input file from which no output would normally have been printed. The scanning will stop on the first match." [11:41] "Only the names of files not containing selected lines are written to standard output. Pathnames are listed once per file searched. If the standard input is searched, the string ``(standard input)'' is written." [11:42] unless I'm misunderstanding something, I don't think it sounds relevant [11:46] the part of the command you gave before the pipe doesn't make a lot of sense to me (I'm not familiar with the parallel command, nor am I familiar with ":::" or the use of "/**/" as opposed to "/*/") [11:49] my guess is that using grep -L would give you the filenames of files with a line that doesn't contain OK (but not the contents of that line) [11:49] redyoshi4, oh, I'm grepping over the output of untar t, not the contents of the files [11:50] and /**/ is a nice bit of ZSH syntax, allows you to glob recursively [11:50] ah, ZSH [11:50] that'd be why I didn't recognize it [11:50] I only know bash [11:59] *** lbft has quit IRC (Read error: Operation timed out) [11:59] *** lbft has joined #archiveteam [12:01] ...I think it's bedtime for me; I stayed up /way/ too late... [12:01] have a good night [12:01] *** redyoshi4 has quit IRC (Quit: Page closed) [12:14] *** VADemon has joined #archiveteam [12:20] Wow, this is going to be a big project http://madrotter-treasure-hunt.blogspot.co.uk/ [12:21] 2000+ links 160+ GB and counting [12:21] amazingly they're all functional! [12:42] *** sankin has joined #archiveteam [12:50] *** Jogie has quit IRC (ZNC - http://znc.in) [12:55] *** xyzzy has joined #archiveteam [12:59] *** berndj has quit IRC (Ping timeout: 369 seconds) [13:06] http://www.jcmit.com/disk2014.htm [13:12] *** Emcy has quit IRC (Read error: Connection reset by peer) [13:12] *** Emcy has joined #archiveteam [13:14] *** MMovie2 has joined #archiveteam [13:15] *** habi has joined #archiveteam [13:15] going to pay for mediafire premium to make this thing go faster.. [13:16] *** MMovie has quit IRC (Ping timeout: 306 seconds) [13:25] is using a torrent the best way to do resumable uploads to IA? [13:27] johtso: are you using plowshare or jdownloader? [13:27] balrog, jdownloader [13:28] thinking of spinning up a server and doing it on that for faster upload/download [13:28] I think plowshare still works [13:29] if you provide me with a list of links I can run something [13:29] *** habi has quit IRC (Quit: Leaving.) [13:29] right now I'm archiving the luth.se ftp [13:29] which has a bunch of mirrors which are now nowhere else [13:29] and which is often down [13:41] balrog, these are all the links https://gist.githubusercontent.com/johtso/415164691861176fce74/raw/0ebf1f5e054f270bb1c6a0e3d9212cc49177926b/gistfile1.txt [13:41] want to split this? [13:42] sounds like it might be a good idea [13:43] balrog, how about I get started on it, see what kind of speeds I'm getting, and then ping you if it's looking unmanageable? [13:43] Thanks for being an Amazon Cloud Drive customer! Amazon Cloud Drive is introducing new unlimited storage plans. [13:43] You can now save all your photos in one place with the Unlimited Photos plan for $11.99/year, plus 5 GB for videos and files. [13:47] *** Start has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** underscor has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** pikhq has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** useretail has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:47] *** will has quit IRC (ircd.shaw.ca irc.shaw.ca) [13:54] *** Jonimus has joined #archiveteam [13:54] *** underscor has joined #archiveteam [13:54] *** wp494 has joined #archiveteam [13:54] *** SadDM has joined #archiveteam [13:54] *** pikhq has joined #archiveteam [13:54] *** useretail has joined #archiveteam [13:54] *** matthusby has joined #archiveteam [13:54] *** will has joined #archiveteam [13:54] *** irc.shaw.ca sets mode: +oo underscor SadDM [13:54] *** swebb sets mode: +o underscor [13:54] *** swebb sets mode: +o SadDM [14:10] johtso: that's reasonable [14:10] I have 50/50 internet, FWIW [14:12] jealous! [14:22] joepie91_: unlimited for pictures only? [14:23] *** arb735 has joined #archiveteam [14:25] *** arb735 has quit IRC (Client Quit) [14:30] *** mistym has joined #archiveteam [14:37] *** Start has joined #archiveteam [14:38] *** Start_ has joined #archiveteam [14:38] *** Start has quit IRC (Read error: Connection reset by peer) [14:57] *** Start_ has quit IRC (Disconnected.) [14:59] *** Start has joined #archiveteam [15:00] *** Start_ has joined #archiveteam [15:00] *** Start has quit IRC (Read error: Connection reset by peer) [15:16] *** MorbusIff has quit IRC (Quit: http://www.disobey.com/) [15:18] *** Morbus has joined #archiveteam [15:28] Simply encode your videos and files into image files... etc. :) [15:28] *** VADemon has left ~bye [15:29] *** VADemon has joined #archiveteam [15:31] *** philpem has joined #archiveteam [15:51] *** Start_ has quit IRC (Disconnected.) [16:01] *** Start has joined #archiveteam [16:02] *** mistym has quit IRC (Remote host closed the connection) [16:24] panic! [16:24] retrospring is removing everything by 1st [16:27] *** Start has quit IRC (Disconnected.) [16:27] *** Start has joined #archiveteam [16:28] *** Stiletto has quit IRC (Read error: Connection reset by peer) [16:29] *** Stiletto has joined #archiveteam [16:35] *** philpem has quit IRC (Ping timeout: 260 seconds) [16:38] http://archiveteam.org/index.php?title=Retrospring [16:38] do we ArchiveBot this? might need some discovery too. [16:40] *** aaaaaaaaa has joined #archiveteam [16:45] *** Start has quit IRC (Disconnected.) [16:49] *** mistym has joined #archiveteam [16:54] *** patricko- is now known as patrickod [17:02] *** patrickod is now known as patricko- [17:18] *** habi has joined #archiveteam [17:27] *** habi has quit IRC (Quit: Leaving.) [17:28] *** brayden has quit IRC (Read error: Connection reset by peer) [17:40] *** patricko- is now known as patrickod [17:43] *** brayden has joined #archiveteam [17:50] *** patrickod is now known as patricko- [18:21] *** patricko- is now known as patrickod [18:23] *** ats has quit IRC (Read error: Operation timed out) [18:23] *** ats has joined #archiveteam [18:26] what would people's preferred servers be for spinning up temporarily and doing some data wrangling (downloading a couple of hundred GB and uploading to IA)? [18:27] was thinking digitalocean for simplicity and the fact they don't charge for data transfer [18:27] but the scaling of storage space with price is a bit annoying [18:28] could go with classic ec2, but I'd rather not pay transfer fees [18:29] would be nice if IA had some ec2 servers so you could transfer to them for free [18:36] they expose an S3 API [18:39] *** patrickod is now known as patricko- [18:45] balrog, but it's not actually on S3 is it? :) [18:46] just an "S3 style" api [18:48] Is there any kind of a IA test server? [18:48] seeing as everything's write only.. [18:48] johtso: IA is somewhere you dump data to [18:49] right? just want to make sure I'm using the tools correctly without making a mess. [18:53] Couple hundred GB might be feasible from home as well :P [18:53] But yes, usually people use some staging machine or server to collect and then upload data. [18:54] *** godane has quit IRC (Ping timeout: 306 seconds) [18:54] Only really massive users (like, uploading multiple TB in a month) ended up getting direct access on a IA machine, IIRC [18:55] johtso: what kind of data do you have to download? [18:56] Nemo_bis: mediafire etc. download from blogger sites [18:56] Ah. [18:56] going to have a go with plowshare on a digitalocean instance [18:56] and the IA command line tool [19:00] *** patricko- is now known as patrickod [19:19] *** mistym has quit IRC (Remote host closed the connection) [19:19] *** mistym has joined #archiveteam [19:21] halp [19:21] what do if archivebot is banned [19:25] unban archivebot [19:25] retrospring pls [19:30] *** Start has joined #archiveteam [19:32] *** johtso has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in) [19:36] *** mistym has quit IRC (Remote host closed the connection) [19:36] *** Muad-Dib has quit IRC (Quit: ZNC - http://znc.in) [19:39] *** scyther has joined #archiveteam [19:43] *** Start has quit IRC (Disconnected.) [19:47] *** SN4T14__ has joined #archiveteam [19:53] *** johtso has joined #archiveteam [19:55] *** johtso has quit IRC (Remote host closed the connection) [19:55] *** johtso has joined #archiveteam [19:55] *** SN4T14_ has quit IRC (Ping timeout: 512 seconds) [19:56] *** patrickod is now known as patricko- [20:13] *** johtso has quit IRC (Read error: Connection reset by peer) [20:18] *** johtso has joined #archiveteam [20:19] *** johtso has quit IRC (Remote host closed the connection) [20:28] *** johtso has joined #archiveteam [20:31] *** patricko- is now known as patrickod [20:35] *** mistym has joined #archiveteam [20:38] *** patrickod is now known as patricko- [20:46] *** sankin has quit IRC (Leaving.) [21:37] *** BlueMaxim has joined #archiveteam [21:38] *** patricko- is now known as patrickod [21:39] *** habi has joined #archiveteam [21:42] *** patrickod is now known as patricko- [21:43] *** habi has left [22:15] *** Nertsy has quit IRC (Quit: Nertsy) [22:21] *** Nertsy has joined #archiveteam [22:36] *** scyther has quit IRC (Read error: Connection reset by peer) [22:54] *** Start has joined #archiveteam [23:08] balrog, what IA collection would these blog dumps go into? [23:09] you can only really put them in one collection, and then ask to have them moved afterwards [23:10] dashcloud, urgh, I uploaded from the command line and it put it in the "opensource" collection :/ [23:11] that's fine. [23:11] that's your only choice unless you get granted access to another one [23:11] oh right! [23:11] just paste the link and ask to have it moved [23:11] I thought it was some kind of OSS collection [23:12] Is it possible to disable "derivations"? [23:12] why? [23:12] Or are things only derived after the first file is uploaded anyway, and then you have to explicitly trigger it again? [23:14] the ia command line tool starts a derive task after uploading. [23:14] just if it's not really appropriate.. most of the files being archives that can't have anything interesting done with them, and then a couple of stray unarchived .mp3 files that would then be fussed over [23:14] just because they happened not to be archived [23:14] you might be overthinking this [23:14] maybe it's not an issue [23:14] derivations don't affect the original item at all [23:14] will IA allow duplicate uploads? [23:15] or can I upload with reckless abandon knowing that it wont duplicate data? [23:15] ie uploading the exact same file twice to an item [23:15] what are you trying to do [23:15] ia upload ident file --checksum [23:16] garyrh, ah, perfect [23:16] *** godane has joined #archiveteam [23:16] yipdw, just trying to get to grips with the tools and get familiar with how IA works [23:16] and not make too much of a mess :) [23:17] oh [23:18] yipdw, I'm doing some semi manual rescuing of file locker downloads linked to from music blogs [23:19] this being my first target http://madrotter-treasure-hunt.blogspot.co.uk/ [23:19] ~2k links 160GB [23:19] oh, ok, yeah, one item per site is probably site [23:19] er fine [23:20] ia upload accepts multiple files and can resume [23:20] any way to make it go faster, or should I just be patient? :) [23:20] it will use whatever bandwidth you allocate to it [23:21] *** matthusby has quit IRC (Ping timeout: 370 seconds) [23:21] when uploading twitpic data I could usually achieve at least ~5-10 MB/s, sometimes more [23:22] ah, awesome [23:24] hah, this is such a hacky setup.. juggling xvfb and vnc to get jdownloader running [23:25] it's a shame plowshare doesn't support more hosts [23:25] jdownloader just seems to be the most complete/robust tool out there [23:31] huh, it's decided it's a community text now? [23:31] that's the default collection iirc. [23:31] oh, that's the same thing as opensource [23:32] seems a bit confusing [23:32] *** matthusb- has joined #archiveteam [23:41] *** xtr-201 has joined #archiveteam [23:53] *** Jogie has joined #archiveteam