[03:08] rather quiet tonight. no-one must be drinking. [03:18] I am working on my new backup [03:20] I'm drinking... which is why I'm quiet... dumb enough to try and play musical chairs with linux partitions [03:24] i just bought 180gb of credit for my usenet acccount [03:24] * Aranje can't watch as the new / is expanded to cover the old data [03:25] It took, now to see if the fucker boots! [03:51] Damn people who get their traceroutes in order.. [04:26] godane, I get completely free Usenet from my ISP but I've never gotten around to actually looking at what Usenet is. Do you mind explaining it to me? [04:35] usenet is the precursor to what we know as forums [04:36] made up of tens of thousands of groups ranging from mysql to cooking [04:36] It supports threaded conversations and attachments [04:36] in 1980 [04:37] stackoverflow is the next iteration. A moderated Q&A targeted at a small market [04:41] The problem with usenet is that most was unmoderated so the signal to noise sucked if you didn't know how to navigate it [04:42] sure sounds like it'd be that way [04:44] many of the usenet groups had concentrations of expertise that far eclipse any modern forum [04:44] linux kernel development started there [04:44] gnu [04:44] emacs [04:44] imdb [04:44] fanfiction.net [04:44] the servers traded news feeds for free [04:45] We see stuff from comp.lang.c and comp.lang.lisp still mentioned in blog posts today [04:45] When the internet got too popular is when some say usenet started to loose importance [04:47] is it anywhere near relevant today? [04:47] I can honestly say as a whole the internet was more civilized. [04:47] well most of it is sucked in to build google groups [04:47] and a ton of people still use that [04:48] many universities still run newsgroups internally like a bbs [04:49] It is all trade offs [04:50] google groups? [04:50] yeah they bought a usenet provider to build that [04:50] trying to compete with yahoo groups [04:50] which are both just more controlled versions of usenet [04:52] fair enough [04:52] is it worth hopping on usenet these days? [04:52] it really depends on which groups you follow [04:53] there are currently 111,000 groups [04:54] jesus [04:57] how would I take a look around? client? [05:00] played musical chairs with linux partitions... grub held a temporary victory, but I won the war. [05:10] BlueMax: usenet was originally for discussion as explained above, but at some point it was figured out that you could post files on it as well if you encoded them properly [05:11] nowadays the file sharing side of it is much more popular than the rest [05:11] except the dmca is taking that apart slowly [05:12] damn americans. [05:12] I fucking hate that law [05:12] it's been traditional for pirate groups of tv shows, movies, games, etc. to do the initial posting on usenet, after which it gets reposted to torrent sites etc. [05:12] some groups did irc first [05:12] ftp too [05:13] ISPs provide usenet free access generally have a poor service in comparison with the paid services like godane is talking about [05:13] e.g. slow download speeds and only keeping a month's worth of history [05:13] Are there even any pay usenet services outside the US? [05:14] I think the provider my ISP uses is GigaNews [05:14] they any good? [05:14] that's one of the big name ones yes [05:14] I wonder how much they provide through the ISP deal though [05:15] apparently it's infinite. [05:15] probably not the bin groups [05:15] yeah that goes back to the asshat AG from New York [05:15] up till that lawsuit every isp carried everything [05:16] Asshat AG? [05:16] attorney general probably [05:16] the Attorney General [05:16] fucker did the case to get in the news which it did [05:17] had a good half year run of news coverage [05:17] Like most things that fuck the internet it comes down to a clown trying to make a name [05:17] Jack Thompson [05:17] for example [05:18] The history of the internet can be summed up as a large stack of lawsuits [05:19] and terrible IP law in the usa [05:19] I hate US copyright law, it fucks the rest of us more than them [05:19] I am an EFF, creative commons person if it is not blaringly obivious [05:19] No it fucks us hard [05:20] it fucks everyone else trying to force them to be like us [05:20] and no lube [05:20] well it was only a matter of time before someone noticed really, if you look at even just the list of group names they're listing all manner of illegal things like even child porn [05:21] That was the argument used to try and shut usenet down [05:21] figures [05:22] and it will be used again to try and shut other things down because it sounds good in the media [05:22] we are STOPPING child porn [05:22] all they did was move it somewhere else [05:23] I wonder how "file sharing" is going to work now with the six strikes law and vpns [05:23] I find the whole thing fascinating since pirates bring the high tech [05:23] just like porn sites [05:24] look at the current online tv offerings [05:24] there is no where to buy hd files that I can just play [05:24] I gotta stream or get some drm non-hd crap [05:25] movies are only marginally better [05:25] music has somewhat caught up. I bought the soundtrack to TDKR in 24bit flac online [05:26] open source file format and encoder, patent free format, no drm and high quality [05:27] that sounds like heaven. [05:27] oh I was so happy to find that [05:27] Hans Zimmer loves computers, he is all about online [05:28] I'm guessing the movie has nothing like that though [05:29] I didn't look, got the bluray for xmas [05:29] It might be possible [05:29] but the file format will still suck [05:30] Don't you wish all the traditional media marketplaces would just die [05:30] This issue is complex [05:30] Since I cannot buy shows in HD [05:30] I have to get them in bluray [05:31] 1920x1080 vs 1280x720? [05:31] and if that goes away then there is no place to buy stuff you can keep [05:31] no like [05:31] standard def [05:31] I learned all about it reading through the amazon prime docs [05:32] Anyone here heard of oink? [05:32] the sound a pig makes. [05:32] yep [05:32] there's a few people here that're whatters [05:33] I thought that group was the future of the scene but even they got busted [05:34] those sites are the only way to get any good amount of music anymore [05:34] it pisses me off [05:34] Like, I love it, but I do really want to give money to people making music to continue their craft [05:35] I dunno what Oink was [05:35] big fuckin music tracker [05:35] it was high quality [05:36] and had a good community to boot [05:36] yeah the oink community is what made it so good [05:37] I wonder if we are going to go back to just mailing each other physical media [05:37] mail me a tb hd and find out [05:38] actually, preferably 2-3tb hd so others can add stuff too [05:38] haha [05:38] heh [05:39] I have a friend who does that [05:39] she drives from SF to LA often enough that she drops a drive on the way down and picks it back up when she goes back north [05:44] Are there any apps that can copy hulu, amazon, or netflix streams? I haven't looked in over a year. Most stuff I find interesting is on youtube which is easy [05:45] but you know those media companies. Gotta have turf wars [05:47] obvious solution: clandestinely produce a script or app that does exactly that, but for your competitors [05:47] heh [05:47] seriously, our corps suck [05:47] I read a good book where it was part of the background of the world that MMOs did things like that [05:47] Well I have some ideas about it on Linux [05:48] just stick some middleware in to copy the ram [05:48] but since flash is mostly a blackbox it can get tricky [05:48] omf_: that sounds complicated. why not just record the traffic? [05:49] I haven't tested to see if the payloads are encrypted or not [05:49] or if delivered out of order and then pieced together [05:49] I would do both of those things if I was trying to make a secure streaming service [05:52] I am also going on the theory that if it was as simple as capturing the network traffic there would already be apps out there to do it [05:52] take get_flash_video [05:52] it got figured out and then turned into an open source project [05:58] fixed my boot shit, and blasted away my encrypted homedir. the linux, it is MINE [05:59] Aranje: what was the problem? [06:01] dunno, grub couldn't find its ass [06:01] but it boots now, so it's whatever :D [06:01] did you switch to grub 2? [06:01] I was always on grub2 [06:02] okay quick sanity check. My new backup scheme is a set of raid 1 drives. No optical media anymore [06:02] I was playing musical chairs with linux and windows partitions... it didn't work out so well... some UUID's got changed... it was a pain [06:03] but I lost no data, and removed windows from my drive without reinstalling a damn thing [06:04] heh [06:05] omf_: you are going to take one of the mirrored drives out and store it, replacing it with a blank? [06:09] if so, be aware of the BER of the drives [06:09] consumer drives have a bit error rate of 1 in 10^14 [06:10] that's only 11tb [06:10] so if you resilver a 4tb drive, you have a 35% chance that the other drive will misread a bit, and you will lose a sector with no way to recover [06:11] I recommend using ZFS instead. it'll let you do a three-way mirror [06:11] you'll still have the same chance of failure, but a miniscule chance that both drives will exhibit an error in the same sector at the same time [06:11] no the whole raid unit is the backup. Once it is full it gets turned off and put on a shelf [06:11] and since ZFS keeps seperate hashes of every sector it stores, it'll know which drive goofed [06:12] ah [06:12] in that case, make doubly sure that the drives don't get mixed up :) [06:13] I thought about just raid 10 for extra redundancy but I need to earn more money [06:13] also, if you're doing hardware raid, remember that different raid controllers can't read each other's raid drives [06:13] so you'll need spare controllers as well [06:14] That is the advantage to software [06:14] good answer :) [06:15] I want everything to be modular and I found that helps keep the cost down [06:15] a 5 disk enclosure w/ raid 10 costs way more than 3 2 disk enclosures which also support software raid [06:16] yea [06:16] I think I'm about to pull the trigger on a 32tb setup [06:16] home made? [06:17] yea [06:17] for normal usage or backup [06:17] archiving [06:18] a stand alone unit or built into a computer [06:18] a 4u case with 24 drives [06:19] how much power is this thing going to use? [06:21] 300-400 watts [06:22] what kinda drives [06:22] probably needs 750 watt supply to handle the peak load at startup [06:22] I'm thinking of getting Hitachi Ultrastars [06:22] consumer or enterprise grade [06:23] yeah [06:23] cause if you're running any kind of hardware raid, you'll get fucked by consumer drives now or later [06:23] fuckin things time out of the array [06:24] Ultrastar is enterprise, they call the consumer ones Deskstar [06:24] and I'll be using ZFS, so no hardware raid [06:25] heh, cool [06:25] <3 zfs [06:25] solaris or freebsd [06:25] linux [06:25] ooh living on the edge [06:25] well, I might give IllumOS a try [06:26] I am thinking about building up a firewall with a raspberry pi [06:26] ZFS on Linux is coming along [06:26] I've had a few problems with it, but nothing that would risk dataloss [06:27] well, except for the bug where chown causes it to lose all the permission information [06:27] sounds like the crap netapp devices used to do [06:28] heh [06:28] then again zfs is just a poor mans https://en.wikipedia.org/wiki/Write_Anywhere_File_Layout [06:28] which netapp owns the patents on [06:29] * Aranje dances around in linux [06:30] even ubuntu is good if you fuck it hard enough [06:30] :D [06:30] ZFS has always seemed more capable than netapp [06:30] it isn't [06:30] there are dozens of things netapp does that zfs doesn't [06:30] such as? [06:31] this is my storage wet dream fs: http://en.wikipedia.org/wiki/HAMMER [06:31] All I know is my college roommate who is a Linux kernel hacker works for netapp writing freebsd kernel shit and he says it is way better. [06:31] in terms of hardware management [06:32] I mean it is a whole stack [06:32] oh, yea [06:32] you get a netapp device you get an embedded freebsd and all this management software [06:32] the in built samba + NFS support [06:32] tuning capability and doing shit while it is online [06:32] ZFS is just the filesystem and raid [06:33] don't get wrong I love zfs because it is pushing this stuff more into the consumer market where it is just as useful [06:33] on solaris they have it dressed up with more apps and shit like a netapp device [06:33] which is why I asked which os you were going to use [06:33] have you given any thought to freenas or that other one [06:34] I always forget the name [06:34] truenas? [06:36] db48x: ZFS via FUSE, or is there actual support for it? [06:36] shaqfu: not ZFS FUSE, ZFS on Linux [06:36] which is an actual kernel module [06:36] !!! [06:36] developed by LLNL [06:36] * shaqfu googles [06:36] http://zfsonlinux.org/ [06:37] I knew they were working on it, but I didn't know it was useful yet [06:38] they have a huge setup already: http://insidehpc.com/2012/04/24/video-sequoias-55pb-lustrezfs-filesystem/ [06:39] That's dynamite [06:40] ZFS on a useful OS; it's like a dream come true [06:43] indeed [06:43] I feel like the hardware isn't quite there though [06:43] there are only so many JBODs I can plug into a single server [06:44] infiniband/fiber channel seem like the better way to go [06:44] just plugging more drives into routers, rather than into HBAs [06:46] that looks like a ragequit [06:48] heh [06:48] I'd like to be able to expand a single set for the rest of my life [06:49] I want to be 5000 years old, capture an asteroid, and use nanotech to convert it into storage and have my zfs pool expand as that comes online [06:49] is that a challenge? :D [06:50] db48x: If only RAIDZ supported expansion :( [06:50] shaqfu: you can replace each drive in the vdev with a larger one until they're all larger [06:50] Aranje: :) [06:51] db48x: True, but as a poor college student, slowly growing my array one drive at a time wasn't an option :) [06:51] Until one day, I slot in the final drive, and poof, 4TB more space [06:55] (that actually would've worked very slowly - Samsung stopped producting 1.5TB drives in that line, so they were sending me 2TB as RMA replacements) [06:57] heh [07:01] yeah here is the new shit [07:01] long term one write media [07:01] http://www.techspot.com/news/50313-hitachi-unveils-quartz-based-storage-data-may-last-100-million-years.html [07:02] 1,000 degrees C for 2 hours and it still works afterward [07:02] This might be the archival storage medium we have all waited for [07:04] omf_: Sure, so long as the devices to read it also last 100M years ;) [07:04] reminds me of http://newscenter.lbl.gov/feature-stories/2009/06/03/billion-year-ultra-dense-memory-chip/ [07:04] the base encoding is binary dots [07:04] so that is good [07:04] here is the funny part [07:05] Los Alamos national lab already built this crystal stuff in 2009 [07:05] they use it now [07:05] nice [07:05] I'd like to buy some [07:05] For what? Long-term storage? [07:05] the really high end research places for hardware get to do some crazy shit [07:05] A medium that actually does not fail [07:06] I have gold gold cdrs I burned in the 90s that still work [07:06] but I cannot buy that media today [07:06] I got 14 year old burned dvdrs that still work and they kinda still sell that quality today [07:06] Oh man, remember when CD-ROM was said to last 500 years [07:06] of course tape drives last a few decades [07:07] in reality the best disks will make it over 100 years and that is about it [07:07] omf_: Long before then, you'll have a loss of reading mechanisms [07:07] yep [07:07] You could have a 8" disk made of adamantium but good luck using it [07:08] See that is the thing [07:08] heh, 40 megs per square inch [07:08] my bluray drive can still play cds [07:08] on the prototype [07:08] they are giving themselves 2 more years [07:08] yea [07:08] omf_: Yeah; backwards compatability lengthens media life considerably [07:09] and that has been intentional [07:09] at least from the computer side [07:09] fuck the media companies [07:09] I am thinking a real long term media like this will get Library of Congress and other bodies approval for use [07:09] meaning there will be continual business for equipment [07:10] But there are still media that were used on dead product genealogies, so to speak - good luck finding GD-ROM drives that aren't built into Sega products [07:10] LC is dealing more with scale than length atm [07:10] was that the sega saturn disks [07:10] Dreamcast [07:11] Length is kinda taken care of, since you can just throw redundancy at a problem [07:11] But moving huge data, that's hard [07:11] I know they're working to figure out how to make Twitter useful [07:12] it depends [07:12] researchers already find twitter useful [07:12] omf_: I mean, getting their dataset to researchers [07:13] yeah cause the dump is too fucking big [07:13] Yep [07:13] shit you cannot even get a copy from the LoC [07:13] I think they'll mail you disks for small (30TB and under?) requests [07:14] You supply the disks [07:14] and then how do they cut the data up for you [07:14] Who knows [07:14] On a smaller scale this is the same kinda problem reddit has [07:15] I wouldn't be surprised if they offer a random section of 1% of Twitter once it's closed [07:15] they have so much "dark" data [07:15] That's still so much fucking data that it'll be statistically significant [07:15] but will it be on topic [07:15] omf_: Most people asking for all of Twitter want to do things like track language development [07:16] well I have interest in doing ngram analysis on that too [07:16] I suppose you could ask for certain subsets, like everything on Election Day, for example [07:16] but to run a dataset like tthat is going to take a computer bigger than what you probably got [07:16] Yeah, you need to be a university [07:16] Take freebase [07:17] I got a whole backup of that locally [07:17] I can only partially load it on a 16gb RAM machine because the server software is fucking java [07:17] and then it runs slow as balls [07:17] You were able to rebuild its functionality? [07:17] largest known graph database in the world [07:18] what do you mean functionality? The graph database does all the work [07:18] Gotcha [07:18] shit they wondered for decades when they would need a graph database [07:18] think of how long the RDBMs have lasted [07:19] csv files, spreadsheets, mysql [07:19] now we got neo4j, graphd, and titan [07:20] multi-threaded, graph systems designed to start at millions of rows and scale in every direction [07:20] sorry it is not even rows [07:21] millions of entities and edges [07:21] Freebase is hundreds of millions facts and categories [07:21] and billions on edges [07:22] Google and Bing both use it to power search now [07:22] it is fun stuff [07:23] the downside is database loading can be measured in days [07:24] I would say there are at least 10 years worth of new apps that can be built upon freebase [07:24] Sheesh [07:25] yeah some jackass did a full freebase load into mysql and it took 2 weeks to load [07:25] while that was a few years ago it is still crazy [07:25] the data is just so complicated but that is the power [07:26] you got metadata galore [07:26] I'd imagine, if you have that much data [07:26] Google bought freebase and all of metaweb [07:26] and it took them a decade to get it viable [07:27] And only in the last year have we seen the results in google and other search engines [07:28] freebase is so big it can identify facebook accounts [07:28] I tried that out and it is crazy [07:28] which means soon it could possibly map everyone on earth [07:29] sweet: scan: scrub repaired 2K in 88h49m with 0 errors on Thu Feb 7 17:39:17 2013 [07:30] I am glad all the data is staying CC [07:30] I am not sure if the data design or the license astounds me more. [07:31] It's good that it's CC - at least it's available to know how you're being mined :P [07:32] I am not sure if the facebook thing is going to be permanent or not [07:32] freebase is about fact based knowledge of the world [07:32] not social people bullshit [07:32] plus the data cannot go in without being CC [07:33] freebase hooks into a bunch of services. They are trying some crazy stuff out [07:33] Wait, is that what Wolfram hooks into? [07:34] They might now [07:34] They never mentioned it when they launched [07:35] I should give it the same test I gave it a year ago [07:36] Hunh, that's unusual [07:38] better response? [07:38] https://www.wolframalpha.com/input/?i=batman&a=*C.batman-_*Movie- [07:39] No - it still doesn't track baseball stats - but the Ty Cobb wiki page had an insane number of hits in January [07:39] no showtimes nearby :( [07:40] yeah stats are tricky because the leagues claim to own them [07:42] That's what I figured [07:42] shit has been going on for years [07:42] it was the first big data thing I was interested in [07:42] Now I am consumed by freebase [07:43] yea, I don't see how they can claim that [07:43] I am building/expanding some existing schemes so I can load a few million facts I collected via running web scrapers for 10+ years [07:43] you can't copyright a fact in the US [07:43] in a phone book, for instance, only the ads have any copyright [07:43] they got a whole racket between the baseball cards and the sports almanic books [07:44] I know that [07:44] It is what it is [07:44] still, logic doesn't enter into it :) [07:44] indeed [07:44] ^.0 [07:45] I guess that is a Teal'C eyebrow raise [07:45] hahaha [07:46] See I think usenet and ircs has helped us evolve the written word [07:46] using emoticons we can transfer feelings and concepts that are far too verbose to be written out constantly [07:47] I mean look at this [07:47] http://www.emojidick.com/ [07:47] It is Moby Dick translated into emoji [07:49] it's not very readable though [07:52] I didn't see a sample chapter on that page [07:55] oh, I was thinking of this: http://languagelog.ldc.upenn.edu/nll/?p=4399 [07:56] but I can't imagine it being much better [07:57] yeah I have seen things like that before [08:29] Still haven't found a good 2 drive enclosure [08:29] reviews are somewhat lacking for this class of device [08:30] heh [08:36] why stop at 2 bays when you can have 16? http://www.ebay.com/itm/SGI-3U-Omnistor-SE3016-SATA-SAS-Expander-Media-Storage-Server-16-Hard-Drive-Bay-/150990090853 [08:43] because I would never want more than a fraction of the drives on anyway [08:43] this is for a cold backup [08:43] since optical media has failed me [08:43]