#archiveteam 2015-03-27,Fri

↑back Search

Time Nickname Message
00:01 🔗 Start letproject newsletter discussion
00:01 🔗 chfoo arkiver: probably not
00:02 🔗 Start let's talk project newsletter on #projectnewsletter
00:02 🔗 Start accidentally hit enter instead of backspace -_-
00:02 🔗 mistym has quit IRC (Remote host closed the connection)
00:03 🔗 arkiver chfoo: we put it in archivebot, what was discovered
00:03 🔗 * arkiver is afk for the night, good day/night all!!
00:08 🔗 dashcloud has quit IRC (Read error: Operation timed out)
00:18 🔗 dashcloud has joined #archiveteam
00:52 🔗 joepie91_ "from march 29, this device no longer supports Music Unlimited"
00:52 🔗 joepie91_ fuck off
00:52 🔗 joepie91_ my device supports it just fine
00:52 🔗 joepie91_ YOU don't support it
00:52 🔗 garyrh has joined #archiveteam
00:54 🔗 xmc basically.
00:55 🔗 Ymgve has quit IRC ()
01:01 🔗 mistym has joined #archiveteam
01:38 🔗 mistym has quit IRC (Remote host closed the connection)
01:39 🔗 mistym has joined #archiveteam
01:57 🔗 mistym_ has joined #archiveteam
02:02 🔗 mistym has quit IRC (Read error: Operation timed out)
02:10 🔗 VADemon has quit IRC (Read error: Connection reset by peer)
02:16 🔗 Stiletto has joined #archiveteam
03:24 🔗 primus104 has quit IRC (Leaving.)
03:38 🔗 mistym_ has quit IRC (Remote host closed the connection)
03:39 🔗 SN4T14_ has quit IRC (Read error: Connection reset by peer)
03:39 🔗 Emcy has quit IRC (Read error: Connection reset by peer)
03:39 🔗 Emcy has joined #archiveteam
03:39 🔗 SN4T14_ has joined #archiveteam
03:42 🔗 SN4T14_ has quit IRC (Read error: Connection reset by peer)
03:42 🔗 SN4T14_ has joined #archiveteam
04:04 🔗 mistym has joined #archiveteam
04:05 🔗 aaaaaaaaa has quit IRC (Leaving)
04:26 🔗 Nertsy has quit IRC (Quit: Nertsy)
04:29 🔗 Nertsy has joined #archiveteam
05:14 🔗 cadbury_ has quit IRC (Read error: Connection reset by peer)
05:15 🔗 aNthraXx_ has joined #archiveteam
05:15 🔗 aNthraXx has quit IRC (Read error: Connection reset by peer)
05:15 🔗 cadbury_ has joined #archiveteam
05:24 🔗 JMC_ is now known as JMC
06:42 🔗 redyoshi4 has joined #archiveteam
06:43 🔗 redyoshi4 hey, I'd like to make an account on archiveteam.org's wiki so as to add a site to the deathwatch list, and I need the secret word for account creation
06:44 🔗 redyoshi4 WHAT FORSOOTH, PRITHEE TELL ME THE SECRET WORD
06:44 🔗 redyoshi4 (in case the exact text is necessary for that)
06:45 🔗 garyrh yahoosucks
06:45 🔗 redyoshi4 that's the secred word?
06:45 🔗 garyrh yep
06:45 🔗 redyoshi4 *secret
06:45 🔗 redyoshi4 OK
06:46 🔗 qwebirc77 has joined #archiveteam
06:46 🔗 qwebirc77 bah; net connection dropped on me
06:46 🔗 qwebirc77 thanks
06:47 🔗 qwebirc77 (I was redyoshi4)
06:50 🔗 redyoshi4 has quit IRC (Ping timeout: 240 seconds)
07:08 🔗 qwebirc77 is now known as redyoshi4
07:09 🔗 mistym has quit IRC (Remote host closed the connection)
07:17 🔗 mistym has joined #archiveteam
07:25 🔗 SimpBrain has joined #archiveteam
07:28 🔗 mistym has quit IRC (Read error: Operation timed out)
07:29 🔗 redyoshi4 http://archiveteam.org/index.php?title=Deathwatch#FurAffinity <- I finished making my entry; to double check, is it in the right place?
08:24 🔗 dashcloud has quit IRC (Read error: Operation timed out)
08:30 🔗 dashcloud has joined #archiveteam
08:36 🔗 schbirid has joined #archiveteam
09:35 🔗 Ymgve has joined #archiveteam
10:23 🔗 signius has quit IRC (Read error: Operation timed out)
10:30 🔗 BlueMaxim has quit IRC (Quit: Leaving)
10:33 🔗 habi has joined #archiveteam
10:35 🔗 signius has joined #archiveteam
10:37 🔗 Mayonaise has quit IRC (Ping timeout: 512 seconds)
10:41 🔗 Mayonaise has joined #archiveteam
10:47 🔗 habi has left
10:56 🔗 johtso this is a rather nice command.. parallel unrar t ::: ./**/*.rar | grep -v OK
10:56 🔗 johtso wonder if there's a nicer way to get clear feedback on exactly which archives failed the test though
10:56 🔗 johtso there must be some combination of flags that would make things clearer
11:39 🔗 redyoshi4 johtso: maybe you want the -L flag from the grep man page?
11:40 🔗 redyoshi4 "Suppress normal output; instead print the name of each input file from which no output would normally have been printed. The scanning will stop on the first match."
11:41 🔗 johtso "Only the names of files not containing selected lines are written to standard output. Pathnames are listed once per file searched. If the standard input is searched, the string ``(standard input)'' is written."
11:42 🔗 johtso unless I'm misunderstanding something, I don't think it sounds relevant
11:46 🔗 redyoshi4 the part of the command you gave before the pipe doesn't make a lot of sense to me (I'm not familiar with the parallel command, nor am I familiar with ":::" or the use of "/**/" as opposed to "/*/")
11:49 🔗 redyoshi4 my guess is that using grep -L would give you the filenames of files with a line that doesn't contain OK (but not the contents of that line)
11:49 🔗 johtso redyoshi4, oh, I'm grepping over the output of untar t, not the contents of the files
11:50 🔗 johtso and /**/ is a nice bit of ZSH syntax, allows you to glob recursively
11:50 🔗 redyoshi4 ah, ZSH
11:50 🔗 redyoshi4 that'd be why I didn't recognize it
11:50 🔗 redyoshi4 I only know bash
11:59 🔗 lbft has quit IRC (Read error: Operation timed out)
11:59 🔗 lbft has joined #archiveteam
12:01 🔗 redyoshi4 ...I think it's bedtime for me; I stayed up /way/ too late...
12:01 🔗 redyoshi4 have a good night
12:01 🔗 redyoshi4 has quit IRC (Quit: Page closed)
12:14 🔗 VADemon has joined #archiveteam
12:20 🔗 johtso Wow, this is going to be a big project http://madrotter-treasure-hunt.blogspot.co.uk/
12:21 🔗 johtso 2000+ links 160+ GB and counting
12:21 🔗 johtso amazingly they're all functional!
12:42 🔗 sankin has joined #archiveteam
12:50 🔗 Jogie has quit IRC (ZNC - http://znc.in)
12:55 🔗 xyzzy has joined #archiveteam
12:59 🔗 berndj has quit IRC (Ping timeout: 369 seconds)
13:06 🔗 Nemo_bis http://www.jcmit.com/disk2014.htm
13:12 🔗 Emcy has quit IRC (Read error: Connection reset by peer)
13:12 🔗 Emcy has joined #archiveteam
13:14 🔗 MMovie2 has joined #archiveteam
13:15 🔗 habi has joined #archiveteam
13:15 🔗 johtso going to pay for mediafire premium to make this thing go faster..
13:16 🔗 MMovie has quit IRC (Ping timeout: 306 seconds)
13:25 🔗 johtso is using a torrent the best way to do resumable uploads to IA?
13:27 🔗 balrog johtso: are you using plowshare or jdownloader?
13:27 🔗 johtso balrog, jdownloader
13:28 🔗 johtso thinking of spinning up a server and doing it on that for faster upload/download
13:28 🔗 balrog I think plowshare still works
13:29 🔗 balrog if you provide me with a list of links I can run something
13:29 🔗 habi has quit IRC (Quit: Leaving.)
13:29 🔗 balrog right now I'm archiving the luth.se ftp
13:29 🔗 balrog which has a bunch of mirrors which are now nowhere else
13:29 🔗 balrog and which is often down
13:41 🔗 johtso balrog, these are all the links https://gist.githubusercontent.com/johtso/415164691861176fce74/raw/0ebf1f5e054f270bb1c6a0e3d9212cc49177926b/gistfile1.txt
13:41 🔗 balrog want to split this?
13:42 🔗 johtso sounds like it might be a good idea
13:43 🔗 johtso balrog, how about I get started on it, see what kind of speeds I'm getting, and then ping you if it's looking unmanageable?
13:43 🔗 joepie91_ Thanks for being an Amazon Cloud Drive customer! Amazon Cloud Drive is introducing new unlimited storage plans.
13:43 🔗 joepie91_ You can now save all your photos in one place with the Unlimited Photos plan for $11.99/year, plus 5 GB for videos and files.
13:47 🔗 Start has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 Jonimus has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 underscor has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 wp494 has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 SadDM has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 pikhq has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 useretail has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 matthusby has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:47 🔗 will has quit IRC (ircd.shaw.ca irc.shaw.ca)
13:54 🔗 Jonimus has joined #archiveteam
13:54 🔗 underscor has joined #archiveteam
13:54 🔗 wp494 has joined #archiveteam
13:54 🔗 SadDM has joined #archiveteam
13:54 🔗 pikhq has joined #archiveteam
13:54 🔗 useretail has joined #archiveteam
13:54 🔗 matthusby has joined #archiveteam
13:54 🔗 will has joined #archiveteam
13:54 🔗 irc.shaw.ca sets mode: +oo underscor SadDM
13:54 🔗 swebb sets mode: +o underscor
13:54 🔗 swebb sets mode: +o SadDM
14:10 🔗 balrog johtso: that's reasonable
14:10 🔗 balrog I have 50/50 internet, FWIW
14:12 🔗 johtso jealous!
14:22 🔗 Rotab joepie91_: unlimited for pictures only?
14:23 🔗 arb735 has joined #archiveteam
14:25 🔗 arb735 has quit IRC (Client Quit)
14:30 🔗 mistym has joined #archiveteam
14:37 🔗 Start has joined #archiveteam
14:38 🔗 Start_ has joined #archiveteam
14:38 🔗 Start has quit IRC (Read error: Connection reset by peer)
14:57 🔗 Start_ has quit IRC (Disconnected.)
14:59 🔗 Start has joined #archiveteam
15:00 🔗 Start_ has joined #archiveteam
15:00 🔗 Start has quit IRC (Read error: Connection reset by peer)
15:16 🔗 MorbusIff has quit IRC (Quit: http://www.disobey.com/)
15:18 🔗 Morbus has joined #archiveteam
15:28 🔗 antomatic Simply encode your videos and files into image files... etc. :)
15:28 🔗 VADemon has left ~bye
15:29 🔗 VADemon has joined #archiveteam
15:31 🔗 philpem has joined #archiveteam
15:51 🔗 Start_ has quit IRC (Disconnected.)
16:01 🔗 Start has joined #archiveteam
16:02 🔗 mistym has quit IRC (Remote host closed the connection)
16:24 🔗 Sanqui panic!
16:24 🔗 Sanqui retrospring is removing everything by 1st
16:27 🔗 Start has quit IRC (Disconnected.)
16:27 🔗 Start has joined #archiveteam
16:28 🔗 Stiletto has quit IRC (Read error: Connection reset by peer)
16:29 🔗 Stiletto has joined #archiveteam
16:35 🔗 philpem has quit IRC (Ping timeout: 260 seconds)
16:38 🔗 Sanqui http://archiveteam.org/index.php?title=Retrospring
16:38 🔗 Sanqui do we ArchiveBot this? might need some discovery too.
16:40 🔗 aaaaaaaaa has joined #archiveteam
16:45 🔗 Start has quit IRC (Disconnected.)
16:49 🔗 mistym has joined #archiveteam
16:54 🔗 patricko- is now known as patrickod
17:02 🔗 patrickod is now known as patricko-
17:18 🔗 habi has joined #archiveteam
17:27 🔗 habi has quit IRC (Quit: Leaving.)
17:28 🔗 brayden has quit IRC (Read error: Connection reset by peer)
17:40 🔗 patricko- is now known as patrickod
17:43 🔗 brayden has joined #archiveteam
17:50 🔗 patrickod is now known as patricko-
18:21 🔗 patricko- is now known as patrickod
18:23 🔗 ats has quit IRC (Read error: Operation timed out)
18:23 🔗 ats has joined #archiveteam
18:26 🔗 johtso what would people's preferred servers be for spinning up temporarily and doing some data wrangling (downloading a couple of hundred GB and uploading to IA)?
18:27 🔗 johtso was thinking digitalocean for simplicity and the fact they don't charge for data transfer
18:27 🔗 johtso but the scaling of storage space with price is a bit annoying
18:28 🔗 johtso could go with classic ec2, but I'd rather not pay transfer fees
18:29 🔗 johtso would be nice if IA had some ec2 servers so you could transfer to them for free
18:36 🔗 balrog they expose an S3 API
18:39 🔗 patrickod is now known as patricko-
18:45 🔗 johtso balrog, but it's not actually on S3 is it? :)
18:46 🔗 johtso just an "S3 style" api
18:48 🔗 johtso Is there any kind of a IA test server?
18:48 🔗 johtso seeing as everything's write only..
18:48 🔗 balrog johtso: IA is somewhere you dump data to
18:49 🔗 johtso right? just want to make sure I'm using the tools correctly without making a mess.
18:53 🔗 Nemo_bis Couple hundred GB might be feasible from home as well :P
18:53 🔗 Nemo_bis But yes, usually people use some staging machine or server to collect and then upload data.
18:54 🔗 godane has quit IRC (Ping timeout: 306 seconds)
18:54 🔗 Nemo_bis Only really massive users (like, uploading multiple TB in a month) ended up getting direct access on a IA machine, IIRC
18:55 🔗 Nemo_bis johtso: what kind of data do you have to download?
18:56 🔗 johtso Nemo_bis: mediafire etc. download from blogger sites
18:56 🔗 Nemo_bis Ah.
18:56 🔗 johtso going to have a go with plowshare on a digitalocean instance
18:56 🔗 johtso and the IA command line tool
19:00 🔗 patricko- is now known as patrickod
19:19 🔗 mistym has quit IRC (Remote host closed the connection)
19:19 🔗 mistym has joined #archiveteam
19:21 🔗 Sanqui halp
19:21 🔗 Sanqui what do if archivebot is banned
19:25 🔗 xmc unban archivebot
19:25 🔗 Sanqui retrospring pls
19:30 🔗 Start has joined #archiveteam
19:32 🔗 johtso has quit IRC (Quit: ZNC - 1.6.0 - http://znc.in)
19:36 🔗 mistym has quit IRC (Remote host closed the connection)
19:36 🔗 Muad-Dib has quit IRC (Quit: ZNC - http://znc.in)
19:39 🔗 scyther has joined #archiveteam
19:43 🔗 Start has quit IRC (Disconnected.)
19:47 🔗 SN4T14__ has joined #archiveteam
19:53 🔗 johtso has joined #archiveteam
19:55 🔗 johtso has quit IRC (Remote host closed the connection)
19:55 🔗 johtso has joined #archiveteam
19:55 🔗 SN4T14_ has quit IRC (Ping timeout: 512 seconds)
19:56 🔗 patrickod is now known as patricko-
20:13 🔗 johtso has quit IRC (Read error: Connection reset by peer)
20:18 🔗 johtso has joined #archiveteam
20:19 🔗 johtso has quit IRC (Remote host closed the connection)
20:28 🔗 johtso has joined #archiveteam
20:31 🔗 patricko- is now known as patrickod
20:35 🔗 mistym has joined #archiveteam
20:38 🔗 patrickod is now known as patricko-
20:46 🔗 sankin has quit IRC (Leaving.)
21:37 🔗 BlueMaxim has joined #archiveteam
21:38 🔗 patricko- is now known as patrickod
21:39 🔗 habi has joined #archiveteam
21:42 🔗 patrickod is now known as patricko-
21:43 🔗 habi has left
22:15 🔗 Nertsy has quit IRC (Quit: Nertsy)
22:21 🔗 Nertsy has joined #archiveteam
22:36 🔗 scyther has quit IRC (Read error: Connection reset by peer)
22:54 🔗 Start has joined #archiveteam
23:08 🔗 johtso balrog, what IA collection would these blog dumps go into?
23:09 🔗 dashcloud you can only really put them in one collection, and then ask to have them moved afterwards
23:10 🔗 johtso dashcloud, urgh, I uploaded from the command line and it put it in the "opensource" collection :/
23:11 🔗 garyrh that's fine.
23:11 🔗 dashcloud that's your only choice unless you get granted access to another one
23:11 🔗 johtso oh right!
23:11 🔗 dashcloud just paste the link and ask to have it moved
23:11 🔗 johtso I thought it was some kind of OSS collection
23:12 🔗 johtso Is it possible to disable "derivations"?
23:12 🔗 garyrh why?
23:12 🔗 johtso Or are things only derived after the first file is uploaded anyway, and then you have to explicitly trigger it again?
23:14 🔗 garyrh the ia command line tool starts a derive task after uploading.
23:14 🔗 johtso just if it's not really appropriate.. most of the files being archives that can't have anything interesting done with them, and then a couple of stray unarchived .mp3 files that would then be fussed over
23:14 🔗 johtso just because they happened not to be archived
23:14 🔗 yipdw you might be overthinking this
23:14 🔗 johtso maybe it's not an issue
23:14 🔗 yipdw derivations don't affect the original item at all
23:14 🔗 johtso will IA allow duplicate uploads?
23:15 🔗 johtso or can I upload with reckless abandon knowing that it wont duplicate data?
23:15 🔗 johtso ie uploading the exact same file twice to an item
23:15 🔗 yipdw what are you trying to do
23:15 🔗 garyrh ia upload ident file --checksum
23:16 🔗 johtso garyrh, ah, perfect
23:16 🔗 godane has joined #archiveteam
23:16 🔗 johtso yipdw, just trying to get to grips with the tools and get familiar with how IA works
23:16 🔗 johtso and not make too much of a mess :)
23:17 🔗 yipdw oh
23:18 🔗 johtso yipdw, I'm doing some semi manual rescuing of file locker downloads linked to from music blogs
23:19 🔗 johtso this being my first target http://madrotter-treasure-hunt.blogspot.co.uk/
23:19 🔗 johtso ~2k links 160GB
23:19 🔗 yipdw oh, ok, yeah, one item per site is probably site
23:19 🔗 yipdw er fine
23:20 🔗 yipdw ia upload accepts multiple files and can resume
23:20 🔗 johtso any way to make it go faster, or should I just be patient? :)
23:20 🔗 yipdw it will use whatever bandwidth you allocate to it
23:21 🔗 matthusby has quit IRC (Ping timeout: 370 seconds)
23:21 🔗 yipdw when uploading twitpic data I could usually achieve at least ~5-10 MB/s, sometimes more
23:22 🔗 johtso ah, awesome
23:24 🔗 johtso hah, this is such a hacky setup.. juggling xvfb and vnc to get jdownloader running
23:25 🔗 johtso it's a shame plowshare doesn't support more hosts
23:25 🔗 johtso jdownloader just seems to be the most complete/robust tool out there
23:31 🔗 johtso huh, it's decided it's a community text now?
23:31 🔗 garyrh that's the default collection iirc.
23:31 🔗 johtso oh, that's the same thing as opensource
23:32 🔗 johtso seems a bit confusing
23:32 🔗 matthusb- has joined #archiveteam
23:41 🔗 xtr-201 has joined #archiveteam
23:53 🔗 Jogie has joined #archiveteam

irclogger-viewer