#archiveteam-ot 2019-09-24,Tue

↑back Search

Time Nickname Message
00:23 πŸ”— Atom-- has joined #archiveteam-ot
00:25 πŸ”— Atom__ has quit IRC (Ping timeout: 252 seconds)
03:09 πŸ”— second Is there a twitch archiver?
03:28 πŸ”— qw3rty has joined #archiveteam-ot
03:38 πŸ”— qw3rty2 has quit IRC (Ping timeout: 745 seconds)
03:42 πŸ”— bluefoo_ has quit IRC (Ping timeout: 492 seconds)
03:58 πŸ”— markedL seems like youtube-dl has some support
04:04 πŸ”— bluefoo has joined #archiveteam-ot
04:34 πŸ”— m007a83 has joined #archiveteam-ot
04:53 πŸ”— ivan_ has quit IRC (Quit: Leaving)
04:54 πŸ”— ivan_ has joined #archiveteam-ot
04:55 πŸ”— Fusl____ sets mode: +o ivan_
04:55 πŸ”— Fusl sets mode: +o ivan_
04:55 πŸ”— Fusl_ sets mode: +o ivan_
04:58 πŸ”— ivan- has joined #archiveteam-ot
04:58 πŸ”— Fusl____ sets mode: +o ivan-
04:58 πŸ”— Fusl sets mode: +o ivan-
04:58 πŸ”— Fusl_ sets mode: +o ivan-
05:09 πŸ”— ivan_ has quit IRC (Ping timeout: 745 seconds)
05:10 πŸ”— ivan- is now known as ivan_
05:19 πŸ”— bluefoo has quit IRC (Read error: Operation timed out)
05:28 πŸ”— kode54 has quit IRC (Quit: The Lounge - https://thelounge.chat)
05:32 πŸ”— killsushi has joined #archiveteam-ot
05:38 πŸ”— kode54 has joined #archiveteam-ot
05:57 πŸ”— hook54321 Last time I tried it on Twitch it had problems if the audio is muted at the start of the stream
06:06 πŸ”— fuzzy8021 has quit IRC (Read error: Connection reset by peer)
06:08 πŸ”— fuzzy8021 has joined #archiveteam-ot
07:01 πŸ”— Raccoon https://news.ycombinator.com/item?id=21052952
07:01 πŸ”— Raccoon 600k Images Removed from ImageNet After Art Project Exposes Racist Bias | Hacker News
07:02 πŸ”— Raccoon https://hyperallergic.com/518822/600000-images-removed-from-ai-database-after-art-project-exposes-racist-bias/
07:23 πŸ”— eythian has quit IRC (Ping timeout: 246 seconds)
07:33 πŸ”— bluefoo has joined #archiveteam-ot
07:47 πŸ”— eythian has joined #archiveteam-ot
08:10 πŸ”— deevious has quit IRC (Quit: deevious)
09:46 πŸ”— deevious has joined #archiveteam-ot
10:11 πŸ”— killsushi has quit IRC (Quit: Leaving)
10:13 πŸ”— betamax lol @ Thomas Cook's last blog post (which they appear to have published AFTER going into liquidation):
10:13 πŸ”— betamax How to Make the Most of Your Family Holiday
10:14 πŸ”— betamax I think the first piece of advice should be "book with someone else, we aren't operating any more"
10:32 πŸ”— kiska18 has quit IRC (Remote host closed the connection)
10:32 πŸ”— kiska18 has joined #archiveteam-ot
10:32 πŸ”— Fusl____ sets mode: +o kiska18
10:32 πŸ”— Fusl sets mode: +o kiska18
10:32 πŸ”— Fusl_ sets mode: +o kiska18
11:01 πŸ”— icedice has joined #archiveteam-ot
11:05 πŸ”— icedice My SSD almost grinds down to a halt sometimes when I try to enter a folder. Everything looks good in CrystalDiskInfo except for ECC Error Rate which is at 200. CrystalDiskInfo still says that the health status is "Good" and "100%" though. The SSD is 1TB and I have 18.8GB free currently (it was down to 16.5GB for a short while before though). Is my SSD going to fail?
11:07 πŸ”— ivan_ icedice: you don't have enough free space, get it to 10% and defragment
11:08 πŸ”— icedice I thought I wasn't supposed to defragment SSDs
11:08 πŸ”— icedice That defragmentation is only for HDDs
11:08 πŸ”— ivan_ is this NTFS?
11:08 πŸ”— icedice Yeah
11:09 πŸ”— icedice Did I permanently damage it?
11:09 πŸ”— ivan_ yeah you might have severe MFT fragmentation
11:09 πŸ”— ivan_ ECC Error Rate doesn't necessarily mean much
11:09 πŸ”— icedice Will it fail?
11:09 πŸ”— ivan_ SSDs can fail suddenly and completely at any time with no warning
11:10 πŸ”— icedice Was the MFT fragmentation message an answer to my question about permanent damage?
11:10 πŸ”— ivan_ https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/defrag
11:10 πŸ”— ivan_ defrag /A C: or whichever drive it is will give you a report
11:10 πŸ”— ivan_ icedice: no
11:11 πŸ”— icedice Thank good
11:11 πŸ”— icedice * God
11:11 πŸ”— icedice And thanks
11:22 πŸ”— icedice Does defrag /A C start the defrag or just give a report?
11:23 πŸ”— ivan_ just a report
11:25 πŸ”— icedice Ok
11:26 πŸ”— icedice "The given volume path is invalid. (0x89000001)"
11:26 πŸ”— icedice It is the C drive though
11:28 πŸ”— icedice Gotta go
11:28 πŸ”— icedice has quit IRC (Quit: Leaving)
11:38 πŸ”— ivan_ someone tell icedice C: is not C
12:13 πŸ”— ShellyRol has quit IRC (Read error: Operation timed out)
12:19 πŸ”— ShellyRol has joined #archiveteam-ot
13:50 πŸ”— second has quit IRC (Remote host closed the connection)
14:03 πŸ”— icedice has joined #archiveteam-ot
14:03 πŸ”— icedice Back
14:13 πŸ”— schbirid2 <@ivan_> someone tell icedice C: is not C
14:14 πŸ”— Raccoon Never hurts to buy a second SSD. I've been having very good success with the Samsung EVO line.
14:14 πŸ”— icedice Ah
14:15 πŸ”— icedice I have a Samsung EVO
14:15 πŸ”— icedice Their great
14:15 πŸ”— icedice I'm just bad at everything I attempt and do
14:15 πŸ”— icedice * They're
14:15 πŸ”— Raccoon What version of windows are you using
14:16 πŸ”— icedice Moving files to my external HDD now
14:16 πŸ”— icedice Windows 10
14:16 πŸ”— icedice It hasn't warned me that I was low on space though
14:16 πŸ”— Raccoon I found Windows 7 automatically did all the things it needed from the get go. So I assume Windows 10 would too
14:16 πŸ”— icedice But I think that's usually when it gets below 10 GB
14:16 πŸ”— icedice And at that point it would probably be too late
14:17 πŸ”— icedice Windows 10 has TRIM
14:17 πŸ”— icedice I'll see if I can activate that manually without having to wait for the automatic monthly TRIM
14:17 πŸ”— Raccoon truth be told, you'll get the most life out of it by keeping it 10 to 50% empty for greater wearleveling distribution.
14:17 πŸ”— icedice I'll go with 20%
14:17 πŸ”— Raccoon if you're going to let media sit, just stick it on a platter drive
14:18 πŸ”— icedice It was all good until I got into Avatar: The Last Airbender and The Legend of Korra
14:18 πŸ”— icedice has quit IRC (Read error: Connection reset by peer)
14:26 πŸ”— icedice has joined #archiveteam-ot
14:27 πŸ”— Raccoon You can't have enough of these. https://www.amazon.com/dp/B0713WPGLL/
14:27 πŸ”— icedice I've probably shortened my SSD's life span by quite a bit, huh?
14:27 πŸ”— Raccoon i doubt it.
14:27 πŸ”— icedice Everything I touch I break
14:27 πŸ”— icedice Oh
14:27 πŸ”— icedice Ok
14:28 πŸ”— icedice That's nice
14:28 πŸ”— Raccoon just do a check disk under properties > tools
14:28 πŸ”— yano_ is now known as yano
14:30 πŸ”— DogsRNice has joined #archiveteam-ot
14:30 πŸ”— Raccoon i personally don't like storing much on SSD and mainly use it for OS and downloads in progress (for speed) and editing (for speed). then files find their final resting place on external platter drives
14:31 πŸ”— Raccoon but once platter drives go away and the price is right, everything will be ssd
14:31 πŸ”— icedice 250 GB free now
14:32 πŸ”— icedice 27.5% free now
14:32 πŸ”— Raccoon nice
14:33 πŸ”— icedice Yeah
14:33 πŸ”— icedice And that's just from moving Avatar: The Last Airbender and The Legend of Korra over
14:35 πŸ”— Raccoon Try these file management programs. Everything (by voidtools), SpaceSniffer, and WizTree. Also FastCopy and dupeGuru and HashCheck Shell Extension (github release)
14:35 πŸ”— icedice I have HashCheck already
14:35 πŸ”— icedice It's pretty nice
14:36 πŸ”— Raccoon I keep a .sha512 of every file over 500 megs
14:36 πŸ”— Raccoon in addition to media collections
14:37 πŸ”— Raccoon so at least you don't have to worry whether you corrupted everything with bitflips everywhere
14:38 πŸ”— icedice I have FreeFileSync
14:38 πŸ”— icedice So it will notice if there are any changes to the files when it compares between my external backup and my internal desktop HDD
14:39 πŸ”— icedice I have this btw:
14:39 πŸ”— icedice https://www.amazon.com/Glyph-BlackBox-BBPR6000-External-Drive/dp/B00Z14R5VM
14:39 πŸ”— icedice https://www.glyphtech.com/product/blackbox-pro
14:39 πŸ”— icedice Hmm
14:40 πŸ”— icedice dupeGuru is a bit like Duplicate Cleaner, I think
14:40 πŸ”— icedice "Windows successfully scanned the drive. No errors were found."
14:44 πŸ”— icedice Any suggestions on a good 10+ TB HDD?
14:45 πŸ”— icedice I met a guy online who has a 10TB cartoon collection in my native language
14:45 πŸ”— icedice Which is pretty rare
14:45 πŸ”— icedice A lot of stuff that was thought to be extinct but people found on recorded VHS tapes
14:48 πŸ”— kiska18 has quit IRC (Read error: Operation timed out)
14:49 πŸ”— kiska18 has joined #archiveteam-ot
14:49 πŸ”— Fusl____ sets mode: +o kiska18
14:49 πŸ”— Fusl sets mode: +o kiska18
14:50 πŸ”— Fusl_ sets mode: +o kiska18
15:02 πŸ”— BlueMax has quit IRC (Quit: Leaving)
15:06 πŸ”— schbirid2 anyone got access to https://www.stan.com.au/watch/spaced ? if so, i would be super grateful for a 1:1 resolution screenshot of any episode
15:16 πŸ”— icedice schibirid2: https://old.reddit.com/r/australia/ and #redditaustralia on Freenode are probably better places to ask
15:24 πŸ”— kiska I would help you, but my stan subscription expired months ago and there isn't anything there to watch for $17/mo
15:33 πŸ”— godane has joined #archiveteam-ot
15:45 πŸ”— markedL are you trying to watch it or archive it?
15:47 πŸ”— markedL it plays / is included in US Amazon Prime Video
15:48 πŸ”— markedL https://www.amazon.com/Art/dp/B07JJ98ZV5
15:52 πŸ”— Stilett0 is now known as Stiletto
15:53 πŸ”— schbirid2 i want to evaluate the quality compared to my dvd before i "archive" it
16:07 πŸ”— markedL do you think the stan version is different than other stream versions?
16:08 πŸ”— markedL https://www.imdb.com/title/tt0187664/episodes?season=1
16:14 πŸ”— schbirid2 no idea
16:22 πŸ”— markedL the imdb one says it's free with account creation
16:29 πŸ”— schbirid2 i dont see that
16:33 πŸ”— markedL https://www.awesomescreenshot.com/image/4250007/3a4b4d8cecce9c11c1b79e86fc95cd69
16:33 πŸ”— markedL what country is your IP in ?
16:36 πŸ”— schbirid2 huh, neat
16:36 πŸ”— schbirid2 de
16:37 πŸ”— schbirid2 but i doubt imdb would have the best quality available or would it?
16:40 πŸ”— markedL hm, well that's what comparisons are for. Wikipedia said that show was original released on Channel 4 at 576i[50]
16:42 πŸ”— markedL and that there were DVD releases in the UK and US
16:48 πŸ”— markedL https://www.channel4.com/programmes/spaced free for UK IP's, and original channel
16:49 πŸ”— icedice Volume Information:
16:49 πŸ”— icedice Volume size = 930,97 GB
16:49 πŸ”— icedice Free space = 393,25 GB
16:49 πŸ”— icedice Total fragmented space = 22%
16:49 πŸ”— icedice Largest free space size = 5,40 GB
16:49 πŸ”— icedice Note: File fragments larger than 64MB are not included in the fragmentation statistics.
16:49 πŸ”— icedice It is recommended that you defragment this volume.
16:50 πŸ”— icedice So, if I use Properties > Tools > Optimize, will that brick my SSD?
17:11 πŸ”— icedice2 has joined #archiveteam-ot
17:13 πŸ”— icedice has quit IRC (Ping timeout: 252 seconds)
17:27 πŸ”— icedice2 TRIM did nothing
17:27 πŸ”— icedice2 defrag /A C: still says the exact same thing
17:35 πŸ”— ScruffyB TRIM just discards unused space. It does not defragment your drive.
17:47 πŸ”— icedice2 Ok
17:47 πŸ”— icedice2 I'm a bit afraid that I'll brick the SSD if I defragment it
17:50 πŸ”— MaximeleG has joined #archiveteam-ot
17:55 πŸ”— icedice2 has quit IRC (Quit: Leaving)
17:56 πŸ”— MaximeleG has quit IRC (Quit: MaximeleG)
17:56 πŸ”— icedice has joined #archiveteam-ot
17:58 πŸ”— markedL isn't TRIM more important than defrag?
18:07 πŸ”— Leslie has left
18:10 πŸ”— markedL https://www.hanselman.com/blog/TheRealAndCompleteStoryDoesWindowsDefragmentYourSSD.aspx
18:20 πŸ”— bluefoo has quit IRC (Ping timeout: 246 seconds)
18:20 πŸ”— schbirid2 its pretty much impossible to brick a ssd by IO unless you stress it in extremely artifical conditions
18:21 πŸ”— schbirid2 any storage can die in an instant, that's just life
18:52 πŸ”— ScruffyB icedice, defragmentation may help a little bit by putting files in contiguous space. Flash drives have very large cluster sizes that have to be deleted and re-written as a group.
18:52 πŸ”— ScruffyB It would speed up writes, not reeds.
18:53 πŸ”— ScruffyB *reads
18:55 πŸ”— ScruffyB Regular defragmentation is not recommened becuase it si a textbok case of write amplification (and flash memory cells have limited write cycles).
18:55 πŸ”— icedice "Storage Optimizer will defrag an SSD once a month if volume snapshots are enabled."
18:55 πŸ”— icedice ^ Yeah, I just found that via StackExchange
18:56 πŸ”— icedice Looks like I might not be shit out of luck after all
18:56 πŸ”— icedice It would be nice if I could see whem the next time Storage Optimizer will run is though
18:59 πŸ”— ScruffyB No option to force it to run immediately?
19:11 πŸ”— icedice I don't know
19:11 πŸ”— Raccoon i mean, what's one write amplification out of the mean time to failure rating of 1 million writes
19:11 πŸ”— icedice Sounds like it's scheduled only, but I'm not sure
19:11 πŸ”— Raccoon defrag to your heart's content. you won't be breathing long enough for the drive to die
19:12 πŸ”— icedice Yeah, but does regular defrags even work on SSDs?
19:12 πŸ”— Raccoon it's just not very as useful to do so, but i wouldn't call it deadly
19:12 πŸ”— kiska Did you say something?
19:12 πŸ”— Raccoon no
19:12 πŸ”— kiska My NVMe drive would like a word: Data Units Written: 475,361,288 [243 TB]
19:13 πŸ”— bluefoo has joined #archiveteam-ot
19:14 πŸ”— Raccoon wouldn't a typical defrag program effectively force data to reshuffle to the least worn (wearleveling) physical sectors
19:15 πŸ”— kiska I just didn't tell you which server its running on. Its an EX42-NVMe from hetzner
20:11 πŸ”— icedice has quit IRC (Ping timeout: 252 seconds)
20:46 πŸ”— schbirid2 has quit IRC (Remote host closed the connection)
20:59 πŸ”— JAA 4.8 TB written each here on the SSDs of my hirola AB pipeline which has only been in operation for a month.
20:59 πŸ”— JAA AB is quite a good way to shred disks.
21:00 πŸ”— JAA Since every response is written to disk three times etc.
21:00 πŸ”— kiska Or being an rsync target is also a good way as well
21:01 πŸ”— kiska Especially during large projects like #googleminus
21:01 πŸ”— JAA Yeah
21:01 πŸ”— kiska Or even the smaller ones like #sketchedout
21:01 πŸ”— JAA The AB pipeline obviously doesn't get anywhere near saturating a Gbit link.
21:02 πŸ”— kiska Or another project *wink* *wink*
21:03 πŸ”— JAA while :; do dd if=/dev/zero of=file bs=16M; rm file; done
21:11 πŸ”— Raccoon why would you write each response to the disk 3 times?
21:12 πŸ”— JAA Because that's how wpull does it and nobody bothered to fix it so far.
21:12 πŸ”— JAA It might actually be something between three and four times, I'm not entirely sure.
21:12 πŸ”— Raccoon isn't that wat caching is for? allow the dumb program to do dumb things, and then flush the final outcome to etch
21:15 πŸ”— JAA So wpull fetches something. The response body is written to disk once for later processing. The full data sent exactly by the server, i.e. with headers and transfer encoding intact, is also written to a temporary file so that it can later be written to the WARC file (after compression, so that's a partial write in terms of data size). And I think it might write the entire body to disk again to a file
21:15 πŸ”— JAA which is deleted again immediately.
21:16 πŸ”— JAA I'm sure about the first three writes (or 2.5 or something depending on the compression ratio).
21:17 πŸ”— JAA I think it should simply write it to disk once with all encoding intact etc., then read it again from that file and processing the transfer encoding etc. on the fly.
21:17 πŸ”— Raccoon what sizes are we talking about
21:18 πŸ”— JAA Uh, whatever the HTTP server sends back?
21:22 πŸ”— Raccoon just imagining whether it'd make more sense for it to strictly use RAM for all that
21:22 πŸ”— Raccoon the way a normal web browser does
21:23 πŸ”— JAA That could work for small responses, but there's no limit on what a server can send back.
21:23 πŸ”— JAA And if you download a file with a browser, it's not kept in memory either.
21:23 πŸ”— Raccoon sure, but then you have a separate job for memory management and such
21:23 πŸ”— Raccoon you get what i mean i hope
21:24 πŸ”— JAA Another problem is that you might not know the response size in the beginning if the server uses chunked TE. So then you'd have to flush everything to disk if a limit is exceeded etc.
21:24 πŸ”— Raccoon but i guess that's why ramdisk exists
21:25 πŸ”— JAA Keeping everything in memory is fine if you know what you're fetching. I do that in qwarc.
21:25 πŸ”— JAA But if you have a tool for the general case of retrieving any URL, it needs to be able to handle large stuff.
21:25 πŸ”— JAA We've had AB pipelines run out of disk space due to big files many, many times.
21:26 πŸ”— JAA Imagine how much worse that would've been if everything was kept in RAM...
22:13 πŸ”— markedL the kernel might not be writing it to disk though it depends on the fs and sync patterns
22:14 πŸ”— JAA Maybe not for small files, yes, but large ones for sure.
22:15 πŸ”— Kaz There’s so need to β€˜well, actually’ your way out of accepting that things could be improved
22:19 πŸ”— ivan_ for files that are written and then soon deleted you might be able to reduce write load with
22:19 πŸ”— ivan_ "vm.dirty_expire_centisecs" = 7 * 60 * 100;
22:19 πŸ”— ivan_ "vm.dirty_writeback_centisecs" = 60 * 100;
22:21 πŸ”— katocala has quit IRC ()
22:25 πŸ”— katocala has joined #archiveteam-ot
22:47 πŸ”— killsushi has joined #archiveteam-ot
23:20 πŸ”— BlueMax has joined #archiveteam-ot
23:51 πŸ”— markedL it bugs me a bit, but lots of businesses were recycled
23:51 πŸ”— markedL Instagram, Facebook, slack
23:53 πŸ”— Raccoon recycled?
23:55 πŸ”— markedL before facebook there was myspace, friendster
23:56 πŸ”— Raccoon right. but it sounded like you said instagram, facebook and slack were recycled
23:56 πŸ”— kiska I think what you mean is the concepts have gone stale
23:57 πŸ”— markedL I mean they were tried before and failed
23:57 πŸ”— markedL before instagram there was moblogging
23:57 πŸ”— Raccoon businesses models fail, technologies not necessarily. it's difficult to monitize something that seems like it should be free
23:57 πŸ”— Raccoon they're all built on the premise that email has been free since at least 1997
23:58 πŸ”— Raccoon or rather, built against that cultural truth
23:59 πŸ”— Raccoon the only thing keeping most of these businesses afloat on their free platforms, are other businesses who are still accustomed to practices like paying for advertisment

irclogger-viewer