[00:08] *** REiN^ has joined #archiveteam-bs [00:28] *** godane has quit IRC (Quit: Leaving.) [00:35] *** BartoCH has joined #archiveteam-bs [01:00] *** Ravenloft has joined #archiveteam-bs [01:35] *** Asparagir has quit IRC (Asparagir) [02:00] *** BlueMaxim has quit IRC (Leaving) [02:00] *** octothorp has quit IRC (Read error: Connection reset by peer) [02:14] *** BlueMaxim has joined #archiveteam-bs [02:25] *** godane has joined #archiveteam-bs [03:53] *** AeonG__ has quit IRC (Ping timeout: 633 seconds) [03:56] *** AeonG__ has joined #archiveteam-bs [03:58] *** Valentine has quit IRC (Ping timeout: 506 seconds) [04:04] *** dashcloud has quit IRC (Read error: Operation timed out) [04:05] *** Odd0002 has quit IRC (Read error: Operation timed out) [04:10] *** BlueMaxim has quit IRC (Leaving) [04:30] *** octothorp has joined #archiveteam-bs [04:43] *** BlueMaxim has joined #archiveteam-bs [04:46] *** qw3rty16 has joined #archiveteam-bs [04:51] *** qw3rty15 has quit IRC (Ping timeout: 600 seconds) [05:02] *** Ravenloft has quit IRC (Ping timeout: 264 seconds) [05:14] *** Odd0002 has joined #archiveteam-bs [06:12] *** wp494 has quit IRC (Ping timeout: 506 seconds) [06:13] *** wp494 has joined #archiveteam-bs [07:02] *** Jusque has quit IRC (Quit: ZNC - http://znc.in) [07:03] *** Jusque has joined #archiveteam-bs [07:06] *** wp494 has quit IRC (Ping timeout: 492 seconds) [07:21] *** wp494 has joined #archiveteam-bs [07:36] *** wp494 has quit IRC (Ping timeout: 633 seconds) [07:39] *** yuitimoth has quit IRC (Ping timeout: 480 seconds) [07:39] *** yuitimoth has joined #archiveteam-bs [07:41] *** wp494 has joined #archiveteam-bs [07:43] *** godane has quit IRC (Read error: Operation timed out) [07:54] *** yuitimoth has quit IRC (Remote host closed the connection) [07:56] *** yuitimoth has joined #archiveteam-bs [07:58] *** Sk1d has quit IRC (Ping timeout: 186 seconds) [08:05] *** Sk1d has joined #archiveteam-bs [08:07] *** wp494 has quit IRC (Read error: Operation timed out) [08:12] *** wp494 has joined #archiveteam-bs [08:34] *** wp494 has quit IRC (Ping timeout: 633 seconds) [08:35] *** wp494 has joined #archiveteam-bs [09:08] *** Stiletto has quit IRC (Ping timeout: 264 seconds) [09:12] *** Stilett0 has joined #archiveteam-bs [09:55] *** BlueMaxim has quit IRC (Leaving) [10:26] *** Stilett0 has quit IRC (Ping timeout: 264 seconds) [10:26] *** Stilett0 has joined #archiveteam-bs [10:53] *** Valentine has joined #archiveteam-bs [11:20] *** Mateon1 has quit IRC (Ping timeout: 250 seconds) [11:20] *** Mateon1 has joined #archiveteam-bs [11:23] *** Ravenloft has joined #archiveteam-bs [13:35] How the F do you spend $2000/month on keeping a forum running? (re SRK tweets/discussion) [13:36] Unless that includes personel cost it sounds absurdly expensive unless your company name is Reddit. [13:39] Reddits costs will be far hgher than that zino [13:40] A good setup of servers will run you that much, AWS with elastic LB'seasily [13:45] The only numbers I could find are from 2010: around $20-30k per month. https://www.reddit.com/r/blog/comments/ctz7c/your_gold_dollars_at_work/c0v8ug8/?context=1 [14:11] *** bwn has quit IRC (Read error: Operation timed out) [14:39] zino: my experience is that almost everybody who goes down due to expenses like that, is running on unnecessarily expensive systems / cloudycloud services [14:39] very rarely are the costs actually justified [14:39] *** bwn has joined #archiveteam-bs [14:39] zino: (the second most common case is running ridiculously non-performant software and throwing endless amounts of servers at it rather than taking a week to optimize it) [14:55] *** icedice has joined #archiveteam-bs [15:01] *** fireglow has joined #archiveteam-bs [15:02] let's hear the bs [15:17] *** icedice has quit IRC (Ping timeout: 260 seconds) [15:19] *** icedice has joined #archiveteam-bs [16:23] joepie91: yeah people forget that scaling isn't only about being able to use more resources when needed, it's also about doing more with less [16:24] low marginal cost is nice, low fixed cost for a small operation is better [16:52] *** schbirid has joined #archiveteam-bs [17:01] *** schbirid2 has joined #archiveteam-bs [17:05] *** schbirid has quit IRC (Read error: Operation timed out) [17:09] *** sep332_ has quit IRC (konversation out) [17:24] *** schbirid2 has quit IRC (Remote host closed the connection) [17:26] *** schbirid has joined #archiveteam-bs [17:30] *** icedice has quit IRC (Quit: Leaving) [17:33] *** schbirid has quit IRC (Quit: Leaving) [17:40] *** schbirid has joined #archiveteam-bs [17:49] oh ya joepie91 [17:49] when you can run the whole thing in under 200$ off OVH or 2k off AWS [17:50] do burst scaling with AWS and have your core baselines on dedicated hardware [17:50] its super rare to see that [18:11] i am looking a upgrade my NAS, but I am unsure witch way to go [18:12] DIY or off-the-self [18:21] I don't think there's a "right answer", it's whatever you prefer or feel like doing [18:21] I prefer DIY personally [18:22] also cost I guess. I don't know what direction or how much difference there is in cost between off-the-shelf and building from parts [18:24] the diys have better bang per buck [18:24] but the off-the-shelfs are turnkey [18:24] and they cause a bigger bang when they fail :) [18:25] this stuff's a hobby for me so failure is a valuable learning experience :p [18:26] i like the look of the 45drives setups [18:26] so, budget is not a problem? [18:26] if you're willing to drop $(god knows what) on a 45drives box [18:28] *** AeonG__ has quit IRC (Read error: Operation timed out) [18:29] the 15 drive i would like if I could get the rest of the familiy to use it [18:30] ^extended family [18:31] 15 drives can be done for a lot cheaper than $2500 [18:31] depends on the value of your time I guess [18:32] or 8 drive [18:32] if not [18:34] where you go to get a 15 drive nas? [18:34] There are definitely tower cases which can hold 15 drives. I think there's a Lian Li one with 20 slots. [18:34] ebay, tbh [18:35] i have a 16 drive cage i've been looking to unload if yr in seattle [18:36] no australia [18:37] Oh, my deepest condolences. HDD prices over there are even worse than here in Europe. [18:39] that why I looking at upgrading HDD in stages [18:40] "drive-of-the-month club": every time you can afford it, buy the cheapest-per-gigabyte disk you can find, and add it to your array [18:40] you'll mix lots and manufacturers, so they won't all fail at the same time :) [18:41] that what I been doing yearly [18:42] but I hit the port limits [18:46] ah okay [18:48] i like the diy options but i want something that is turnkey just add drives and OS [18:49] my current setup is a eeepc 701SD netbook 4x2TB HDD into 2 RAID arrays and a UPS [18:50] ^usb desktop HHD [19:06] i plan to use http://www.fantec.de/en/products/storage-devices/external-hard-drives/35-inch-external-hard-drives/produkt/details/artikel/1461_fantec_qb_x8us3/ with a raspberry for hoarding [19:06] costs ~230€ [19:10] *** Hiccup has joined #archiveteam-bs [19:11] Is there some kinda of directory or more advanced search engine for the Internet Archive? [19:14] schbirid: :thinking: [19:18] checking on ebay and bookmarked [19:19] AU $922.23 ouch [19:20] Hiccup: https://archive.org/advancedsearch.php [19:21] I wrote a tool specifically for ArchiveBot that lists the files in each item in that collection. [19:21] by more more advanced search I meant one that could search through sub-pages, but I'll try this [19:21] But there is nothing in general to search for filenames etc. [19:21] oh [19:22] (As far as I know, anyway.) [19:23] My tool is at https://github.com/JustAnotherArchivist/archivebot-archives . There's nothing special going on there really; it just uses IA's API to iterate through all items in the collection and writes the file list for each to a YAML file. [19:24] I then just use 'git grep' to search for files (or the GitHub search, because those YAML files are also in the repo). [19:24] But yeah, you basically have to write your own tool which extracts the data you need and run the search locally. [19:49] *** Hiccup has quit IRC (Leaving) [20:11] *** godane has joined #archiveteam-bs [20:12] *** jschwart has joined #archiveteam-bs [20:15] *** BlueMaxim has joined #archiveteam-bs [20:40] *** RichardG has quit IRC (Read error: Operation timed out) [20:47] *** schbirid2 has joined #archiveteam-bs [20:50] *** RichardG has joined #archiveteam-bs [20:50] *** schbirid has quit IRC (Ping timeout: 252 seconds) [21:23] *** schbirid2 has quit IRC (Quit: Leaving) [21:34] *** AeonG__ has joined #archiveteam-bs [22:18] *** jschwart has quit IRC (Quit: Konversation terminated!) [22:50] *** will has quit IRC (Read error: Operation timed out) [22:50] *** will has joined #archiveteam-bs [23:04] i'm starting to upload America Now podcast [23:05] it had Meghan McCain on it [23:06] its about 25gb for 2015 and 2016