#archiveteam 2014-11-28,Fri

↑back Search

Time Nickname Message
00:00 🔗 okeuday1 has quit IRC (Read error: Connection reset by peer)
00:15 🔗 okeuday has joined #archiveteam
00:28 🔗 antomatic has quit IRC (Ping timeout: 265 seconds)
00:31 🔗 antomatic has joined #archiveteam
00:32 🔗 MMovie has joined #archiveteam
00:32 🔗 zenguy_pc has quit IRC (Read error: Operation timed out)
00:34 🔗 MMovie1 has quit IRC (Read error: Operation timed out)
01:17 🔗 Ymgve has quit IRC ()
01:35 🔗 zenguy_pc has joined #archiveteam
01:51 🔗 primus104 has quit IRC (Leaving.)
01:55 🔗 zenguy_pc has quit IRC (Read error: Operation timed out)
02:00 🔗 Morbus has joined #archiveteam
02:03 🔗 zenguy_pc has joined #archiveteam
02:42 🔗 ete_ has quit IRC (Remote host closed the connection)
02:55 🔗 SketchCow Back
02:56 🔗 SketchCow Down to just two directories of verizon warc.gzs
02:56 🔗 SketchCow After that, they need to be bound into megawarcs but all told, this will hopefully be behind us and maybe by the end of the weekend, FOS will function like it used to.
02:56 🔗 SketchCow Maybe.
02:56 🔗 SketchCow It'll certainly have more inodes
03:05 🔗 joepie91 inodes are like herpes, you can never have too many- wait, no, that's not how the saying goes
03:07 🔗 BlueMaxim inodes are like sayings from joepie91 - you can have too many and they might all indeed suck eggs
03:07 🔗 joepie91 :(
03:12 🔗 BlueMaxim I dunno what it is about sucking eggs I just love using the phrase
03:17 🔗 Jonimus has quit IRC (Quit: WeeChat 1.0.1)
03:21 🔗 Jonimus has joined #archiveteam
03:30 🔗 mistym has joined #archiveteam
03:34 🔗 godane you guys will be starting to get radionz mp3 next week
03:34 🔗 godane cause i'm crazy
03:36 🔗 philpem has quit IRC (Ping timeout: 272 seconds)
04:06 🔗 SketchCow Down to 2 nightmare directories.
04:07 🔗 godane SketchCow: i'm listing to your Boss
04:11 🔗 SketchCow He is very very hoarse
04:21 🔗 godane i noticed that
04:22 🔗 godane after this he can rest his voice i hope
04:36 🔗 godane has quit IRC (Read error: Operation timed out)
04:58 🔗 aaaaaaaaa has quit IRC (Leaving)
05:09 🔗 LordNigh2 has joined #archiveteam
05:11 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
05:12 🔗 dashcloud has joined #archiveteam
05:19 🔗 Lord_Nigh has quit IRC (Ping timeout: 600 seconds)
05:19 🔗 LordNigh2 is now known as Lord_Nigh
05:51 🔗 godane has joined #archiveteam
06:12 🔗 signius has quit IRC (Read error: Operation timed out)
06:25 🔗 signius has joined #archiveteam
07:40 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
07:42 🔗 dashcloud has joined #archiveteam
07:53 🔗 primus104 has joined #archiveteam
08:02 🔗 mistym has quit IRC (Remote host closed the connection)
08:40 🔗 T31M has quit IRC (Quit: Leaving)
08:41 🔗 godane has quit IRC (Ping timeout: 633 seconds)
09:00 🔗 signius has quit IRC (Read error: Operation timed out)
09:59 🔗 Kenshin has quit IRC (Ping timeout: 258 seconds)
09:59 🔗 Kenshin has joined #archiveteam
10:01 🔗 hive-mind has quit IRC (Ping timeout: 272 seconds)
10:02 🔗 hive-mind has joined #archiveteam
10:16 🔗 Kenshin has quit IRC (Ping timeout: 246 seconds)
10:24 🔗 primus104 has quit IRC (Leaving.)
10:25 🔗 Kenshin has joined #archiveteam
10:32 🔗 arkiver SketchCow: cf, computerfreek, has discovered over 1 million viddy items
10:33 🔗 arkiver I'm going to download some of them and get an estimate of the total size
10:34 🔗 arkiver I'm already only downloading the high quality versions of all videos and not the medium and low quality versions
10:40 🔗 BlueMaxim has quit IRC (Quit: Leaving)
10:56 🔗 arkiver SketchCow: with the current 1 million items we have I'm estimating this to be around 2.5 to 3 TB (that's with only the high quality version)
10:56 🔗 arkiver I will also do a grab with all the three version (high, medium, low) and see what the size will be then
11:21 🔗 SN4T14 has quit IRC (Ping timeout: 335 seconds)
11:21 🔗 Ymgve has joined #archiveteam
11:22 🔗 SN4T14 has joined #archiveteam
11:22 🔗 arkiver SketchCow: looks like items are around 60-70% bigger when we download all three video sizes
11:24 🔗 arkiver 50-60%, actually
11:25 🔗 arkiver That makes it around 4-5 TB in size total for the 1 million items we know exist. However, more items might be discovered in the future, which can up the total size by a few TB
11:26 🔗 arkiver I'd say let's do all three versions, but I don't know how the situation on space is at IA, so what do you think we should do?
11:29 🔗 schbirid has joined #archiveteam
11:45 🔗 ohhdemgir has quit IRC (Quit: Leaving)
11:51 🔗 signius has joined #archiveteam
12:38 🔗 midas1 is now known as Midas
12:38 🔗 Midas is now known as midas
13:16 🔗 ex-parro1 has quit IRC (Read error: Operation timed out)
13:19 🔗 ex-parrot has quit IRC (Read error: Operation timed out)
13:23 🔗 ex-parrot has joined #archiveteam
13:23 🔗 ex-parro1 has joined #archiveteam
14:08 🔗 godane has joined #archiveteam
14:14 🔗 philpem has joined #archiveteam
14:47 🔗 ohhdemgir has joined #archiveteam
14:52 🔗 dashcloud has quit IRC (Quit: No Ping reply in 180 seconds.)
14:53 🔗 dashcloud has joined #archiveteam
15:02 🔗 mistym has joined #archiveteam
15:06 🔗 SketchCow Just the largest size.
15:06 🔗 SketchCow Why grab three versions?
15:10 🔗 mistym has quit IRC (Remote host closed the connection)
15:10 🔗 Emcy_ has joined #archiveteam
15:11 🔗 arkiver beecause the medium size is actually used in the webpages, not the high quality size
15:12 🔗 arkiver And I think the more original data we grab from site, the better
15:12 🔗 primus104 has joined #archiveteam
15:14 🔗 SketchCow I think the purpose of this page is middling at best.
15:15 🔗 SketchCow And so I'd rather we ran a backup of the data than have intermediates.
15:15 🔗 SketchCow Someone can generate an intermediate down the line if needed.
15:16 🔗 Emcy has quit IRC (Ping timeout: 480 seconds)
15:16 🔗 SketchCow Down to one nightmare directory left on FOS
15:27 🔗 mistym has joined #archiveteam
15:32 🔗 Emcy has joined #archiveteam
15:35 🔗 Emcy_ has quit IRC (Ping timeout: 480 seconds)
15:36 🔗 aaaaaaaaa has joined #archiveteam
15:58 🔗 Start has joined #archiveteam
16:01 🔗 PepsiMax_ has joined #archiveteam
16:01 🔗 PepsiMax_ has quit IRC (Client Quit)
16:26 🔗 mistym has quit IRC (Remote host closed the connection)
17:02 🔗 dx has quit IRC (Remote host closed the connection)
17:03 🔗 dx has joined #archiveteam
17:15 🔗 mistym has joined #archiveteam
17:18 🔗 mistym has quit IRC (Remote host closed the connection)
17:19 🔗 mistym has joined #archiveteam
17:36 🔗 dxdx has joined #archiveteam
17:36 🔗 dx has quit IRC (Read error: Connection reset by peer)
17:39 🔗 philpem has quit IRC (Ping timeout: 272 seconds)
17:54 🔗 signius has quit IRC (Remote host closed the connection)
17:55 🔗 phuzion has quit IRC (Quit: No Ping reply in 180 seconds.)
17:57 🔗 phuzion has joined #archiveteam
18:04 🔗 signius has joined #archiveteam
18:17 🔗 phuzion has quit IRC (Read error: Operation timed out)
18:17 🔗 phuzion has joined #archiveteam
18:21 🔗 mistym has quit IRC (Remote host closed the connection)
18:43 🔗 godane has quit IRC (Ping timeout: 272 seconds)
18:52 🔗 primus104 has quit IRC (Leaving.)
18:55 🔗 primus104 has joined #archiveteam
18:55 🔗 primus104 has quit IRC (Client Quit)
18:59 🔗 raylee has quit IRC (Quit: ZNC - http://znc.in)
19:02 🔗 mistym has joined #archiveteam
19:10 🔗 godane has joined #archiveteam
19:25 🔗 SketchCow TELE2 project officially done.
19:25 🔗 midas congrads!
19:26 🔗 midas Fos happy again?
19:26 🔗 SketchCow no.
19:26 🔗 midas verizon!
19:26 🔗 SketchCow No, TELE2 was one of three miserable destructions.
19:26 🔗 SketchCow Verizon and Swipnet are the other two.
19:26 🔗 SketchCow The rest, like Halo and Archivebot, generate reasonable amounts of files, and the system isn't in pain.
19:27 🔗 SketchCow I have one directory here. It's going from an rsync'd user upload of verizon accounts to the staging directory that will make the megawarc.
19:27 🔗 SketchCow It has been copying files, just copying them, from one directory to another, for 2 solid days, and counting.
19:30 🔗 midas small files break stuff
19:35 🔗 SketchCow They really do, in this case. FOS is overoaded. I've begun moving some operations to SIS.
19:36 🔗 SketchCow SWIPNET has another 17 packs of material go through.
19:37 🔗 SketchCow VERIZON has four, but those four are unclear on their sizes - I basically had to guess
19:39 🔗 Smiley fr€'~B ~K7~mh,~r2v72
19:39 🔗 SketchCow Exactly. Like that.
19:41 🔗 Smiley SRY, CAT ATTACL
19:42 🔗 godane can some start a archive of this every 5 to 6 days: http://downloads.bbc.co.uk/podcasts/worldservice/newshour/rss.xml
19:42 🔗 Smiley and busted the damn kb ¬_¬; right. Stopping now.
19:42 🔗 godane they delete them after 7 days
19:42 🔗 Smiley godane: yeah they do all their podcasts, I looked at it before
19:42 🔗 Smiley so many tho
19:45 🔗 SketchCow http://i.imgur.com/43fhrpX.gifv
19:49 🔗 Smiley o~_O
19:55 🔗 mistym has quit IRC (Remote host closed the connection)
20:00 🔗 primus104 has joined #archiveteam
20:01 🔗 godane so some good news on radionz
20:02 🔗 godane looks like they start there radio talk year about a month after christmas
20:02 🔗 joepie91 SketchCow: you must be happy that we didn't send Isohunt stuff at FOS :)
20:03 🔗 joepie91 in its original form
20:03 🔗 godane so this means they did just pick everything after jan 21 2008 to delete everything
20:03 🔗 godane so this means the 2008 year maybe full
20:13 🔗 garyrh has quit IRC (Remote host closed the connection)
20:19 🔗 w0rp godane: I wrote a Python 3 script to download MP3s from the RSS feed. https://bpaste.net/show/bb3b816a3c73
20:21 🔗 w0rp Shall I add it to a cronjob to run every once and a while, for how long, and where should the files get uploaded to in the end?
20:21 🔗 w0rp Sorry, not "for how long," how often should it check?
20:24 🔗 nblr_ has joined #archiveteam
20:25 🔗 nblr has quit IRC (Ping timeout: 365 seconds)
20:29 🔗 primus105 has joined #archiveteam
20:30 🔗 joepie91 was the data from Sony ever released?
20:34 🔗 primus104 has quit IRC (Read error: Operation timed out)
20:34 🔗 garyrh has joined #archiveteam
20:46 🔗 slash` has quit IRC (Ping timeout: 480 seconds)
20:54 🔗 ex-parrot has quit IRC (Leaving.)
20:54 🔗 ex-parro1 has quit IRC (Leaving.)
20:54 🔗 ex-parrot has joined #archiveteam
20:55 🔗 ex-parro1 has joined #archiveteam
20:57 🔗 Lord_Nigh has quit IRC (Read error: Operation timed out)
20:59 🔗 Lord_Nigh has joined #archiveteam
21:00 🔗 ruukasu has quit IRC (Quit: WeeChat 1.0.1)
21:04 🔗 Diesel_ has quit IRC (Remote host closed the connection)
21:09 🔗 mistym has joined #archiveteam
21:09 🔗 slash` has joined #archiveteam
21:10 🔗 godane has quit IRC (Ping timeout: 258 seconds)
21:15 🔗 mistym has quit IRC (Remote host closed the connection)
21:17 🔗 mistym has joined #archiveteam
21:19 🔗 ruukasu has joined #archiveteam
21:21 🔗 BlueMaxim has joined #archiveteam
21:26 🔗 BiggieJo1 has quit IRC (Read error: Connection reset by peer)
21:53 🔗 useretail has joined #archiveteam
22:17 🔗 ruukasu has quit IRC (Quit: WeeChat 1.0.1)
22:17 🔗 ruukasu has joined #archiveteam
22:32 🔗 ruukasu has quit IRC (Quit: WeeChat 1.0.1)
22:37 🔗 ruukasu has joined #archiveteam
22:54 🔗 godane has joined #archiveteam
23:04 🔗 mistym has quit IRC (Remote host closed the connection)
23:05 🔗 philpem has joined #archiveteam
23:16 🔗 mistym has joined #archiveteam
23:37 🔗 ex-parro2 has joined #archiveteam
23:48 🔗 ex-parro2 has quit IRC (Remote host closed the connection)

irclogger-viewer