!nostupidquestions is a community dedicated to being helpful and answering each others' questions on various topics.
The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:
Rule 1- All posts must be legitimate questions. All post titles must include a question.
All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.
Rule 2- Your question subject cannot be illegal or NSFW material.
Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.
Rule 3- Do not seek mental, medical and professional help here.
Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.
Rule 4- No self promotion or upvote-farming of any kind.
That's it.
Rule 5- No baiting or sealioning or promoting an agenda.
Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.
Rule 6- Regarding META posts and joke questions.
Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.
On fridays, you are allowed to post meme and troll questions, on the condition that it's in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.
If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.
Rule 7- You can't intentionally annoy, mock, or harass other members.
If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.
Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.
Rule 8- All comments should try to stay relevant to their parent content.
Rule 9- Reposts from other platforms are not allowed.
Let everyone have their own content.
Rule 10- Majority of bots aren't allowed to participate here.
Our breathtaking icon was bestowed upon us by @Cevilia!
The greatest banner of all time: by @TheOneWithTheHair!
👽
When you normally delete a file, it doesn't actually delete it, to save time it just marks the space as free, so any new files can be written into that part of your drive.
But the actual data just remains there until a new file is written to the storage.
SecureErase does the second part without making an actual file.
Normal delete:
File: 01010101 -> no file:01010101
Secure erase:
File: 01010101 -> no file:00000000
That's overwriting on a HDD. secure erase on an SDD throws away the key telling the controller what 01010101 means.
Erase takes the address off your mailbox and leaves the mail in it for the next resident to use. Secure erase opens it and scritches off all the words from your mail. Takes time though.
Perfect ELI5
Your disk is like a file cabinet, there's also an index folder where for example it says that "your file.txt" is in cabinet C7. You go there and there's a sheet of paper written in pencil with the contents of your file. In this analogy here's how several solutions work:
It's worth mentioning that the OS index is what gets deleted when a file is deleted, the SSD is untouched by this, but SecureErase is a special command sent to the SSD controller inside the unit that deletes its own index of what entries correspond to which 1s and 0s on which separate internal NAND flash chips (and dies, planes, blocks and pages) and in what order. This makes recovering data very very difficult even if the actual data is not overwritten in any way, because SSDs will often write in parallel for speed, which renders the oldest most bruteforcry digital forensic technique in the book on HDDs - simply transplanting the platters into a different drive, imaging them into hex and looking for file headers and subsequent zeroes (to identify empty space after a file) and reading in files as if they were written linearly to see what comes out.
That's as I understand it though.
Most of these answers are mostly right: deleting a file on disk doesn't actually erase the data, it just marks the space as available to write over - meaning that so long as nobody's used the space since, you can go retrieve the contents with an undelete utility.
Most of the time, people don't care - but if for instance you're selling the PC or there's highly sensitive information involved, that might not be good enough.
As such, there are utilities that can go out and specifically overwrite the contents of a file with all zeroes, so ensure that it's dead-dead - and there are other utilities that can do the same to an entire disk.
There's one wrinkle: Magnetic HDDs don't reliably erase and overwrite completely in a single pass; just like rubbing out pencil writing, it can leave faint impressions under the new content, and it is actually possible (with serious effort by forensic recovery people) to glean some of the previous content. If there's serious money / security at stake, a simple overwrite is not enough, so there's software that certifiably-randomly scribbles over each bit, seven times over, making the chances of recovering the original astronomically slim. Again, this can be done for individual files or the entire disk.
SSDs aren't prone to leftover impressions, thankfully - what's gone is gone. And they have one other neat feature: while a magnetic disk can only be erased one bit at a time, so large disks can take hours - SSDs can just open the floodgates and ground every cell at once, fully erasing the entire disk in an instant.
This instant-erase, while comprehensive... returns before you've even taken your finger off the ENTER key, so fast it feels like it can't possibly have done anything, it must be broken, how can I trust it? So BIOS manufacturers hype it up, call it something impressive to underline that it's big and powerful, and actually impose a 10-second countdown to make it feel like it's doing something complicated.
Any of these different things have been called 'secure erase' at various points, so it's a little context dependent. But from the end-user perspective: this data is getting shredded then incinerated then added to cattle feed; it's not coming back.
THIS IS THE HILL I DIE ON.
No one has ever recovered overwritten data, as far as anyone can tell. Go look it up. The technique was only a theoretical attack on ancient MFM/RLL hard drive encoding (Gutmann's paper). Even 20 year old drives' (post 2001, approx) magnetic encoding are so small there isn't an 'edge' to read on the bits. A single pass of random data is sufficient to permanently destroy data, even against nation-state level actors. Certainly enough for personal data.
from https://en.wikipedia.org/wiki/Gutmann_method :
Most of the patterns in the Gutmann method were designed for older MFM/RLL encoded disks. Gutmann himself has noted that more modern drives no longer use these older encoding techniques, making parts of the method irrelevant. He said "In the time since this paper was published, some people have treated the 35-pass overwrite technique described in it more as a kind of voodoo incantation to banish evil spirits than the result of a technical analysis of drive encoding techniques"
More reading material:
NOW THAT BEING SAID there is no harm in doing a secure, 35-pass overwrite other than the time, energy and disk wear. If watching all the bit-patterns of a DoD-level wipe using DBAN on a magnetic disk tickles your fancy, or you think this is a CIA misinformation campaign to get people to do something insecure so they can steal your secrets, please just go ahead and do a 35-pass overwrite with alternating bit patterns followed by random data. I can tell you that I believe in my heart-of-hearts, that one pass is sufficient.
This is exactly what a cia analyst whose tan literally comes from their monitors and is never let out of Langley’s 38th sub basement would say.
Guess we’re doing 40 passes. Just to be sure. ;)
In my industry we destroy all storage devices when computers are returned at end of lease, or decommissioned.
When I worked at the e-waste recycle and technomancery place we'd do secure wipes for any hard drives they dropped off with their stuff.
And one time somepne asked if we could do a Gutmann wipe for his hard drive.
His 10TB hard drive.
Interesting. We mostly use DBAN at work because it's a one-button process you can walk away from, and it has drivers for hardware old enough that we're disposing it. Nobody's ever selected the fancy super-paranoid stuff as far as I know.
If the hardware won't boot, we take a layer-1 approach instead :D
There’s one wrinkle: Magnetic HDDs don’t reliably erase and overwrite completely in a single pass; just like rubbing out pencil writing, it can leave faint impressions under the new content, and it is actually possible (with serious effort by forensic recovery people) to glean some of the previous content.
No, it's not. At least it has never been done on more than single bits, with an accuracy higher than 50% (which means even when trying to restore a single byte, your chances of it being correct are 1/256).
ELI5 albeit a long one:
Your storage is like a book, with an index at the beginning. This index tells whatever computer reading it “the contents of “Math Homework 5.odt” start on page 100 and ends on page 125”, for example.
When a new file needs to be stored, the computer only has to quickly read the index, find a spot in the book with empty pages, and write the file on those pages.
Nkw, when you delete a file, the computer will only erase the index, which is much quicker. Math Homework 5.odt is gone so more another file can be written onto pages 100 through 125!
A thief getting their hands on your storage could still read the part with your file's content on it, though, so it's not safe!
“Secure delete” means the computer will not only erase the index entry, but even every letter from every page related to your file's content. This takes a long time, but a thief trying to read those pages will simply see blank pages, so that's why it's safe!
Normal delete just marks things as deleted, freeing up that space to be overwritten later, but the original data is still there until it is overwritten. Secure delete overwrites the data with random 1's and 0's, so it is actually gone and can't be recovered. It's the difference between throwing a physical document into the trash, but not taking the trash out, or instead putting the physical document through the shredder and then burning the shredded paper. In the first example anyone with access can reach into the trash can and read your document, and in the second example it's unrecoverable.
With an HDD, your operating system can (mostly) directly access bits on the magnetic disks, so you can wipe them by just writing 0 to it over and over (historically, there was a paper saying 7 times would make any bits unrecoverable - this changed as density got higher)
With SSDs, your operating system has very little control over what bits a write is touching, a lot more was moved into the firmware on the flash memory itself
So SSDs need a special command “Secure Erase” to wipe them
It's worth it to mention that after a single pass it was possible to recover single bits with an electron microscope, but not even a full byte. One pass has always been enough to delete actually meaningful data.
Yeah, and as densities have increased, fewer passes have been needed to even do that
It depends on the who made it.
If your hard drives are encrypted then it doesn't delete the data, it deletes the encryption key which makes the data effectively unrecoverable, while others will take the existing ones and zeros of data and turn them all into ones, then turn them all into zeros and repeat 3 or 4 times just to be sure.
One thing other answers have missed is that some ssds encrypt data before writing and obviously after reading (this prevents a swap the storage controller type attack) A secure erase on such a device consists of changing the read/write key. Takes milliseconds. Irrevocable (unless you find a way to read previous contents of the key storage)
Hot take: It’s what old farts used to use back in the day.
These days you should just use encrypting file systems with per-file keying (like eg APFS) and stop worrying about deletion leaving file data around.
I don't want to encrypt my drive in case I forget the encryption key and lose all my data
Yeah, I'm having a hard time deciding about that.
On the one hand I'm often in business trips and want to be at least able to recover my data easily if something happens.
On the other hand getting the laptop stolen with all the data is a nightmare.
Didn't look into current solutions for quite some time though. As I'm currently waiting for a new laptop, this could be a good time to check something out.
Anyone with some suggestions? (Will look into the encrypted FS OP recommended, but having options and opinions never hurts)
Edit: ok, just realised that APFS is Apple File System. So that's not gonna roll for me, as I'm on Linux and would like to have something that's actually designed and developed for it. Don't know anything about Linux APFS support, but I'm a bit traumatised by NTFS and anyway I don't want to have anything to do with Apple
Anyone with experience with encrypted LUKS and data recovery?
Some buzzwords for options would also greatly help me
Thanks for anything in advance!
3-2-1 backup is important. I've been burned with lost files before, so I now make sure they're available in multiple places.
I also encrypt everything. My laptops can't be unlocked by anyone except myself: Apple Filevault on my Apple laptop, LVM on LUKS on my Linux laptop. If something happens to me, my laptops must be wiped completely to be useable as a used device.
My NAS keeps my backups of all my documents and media (and as a hobbyist photographer, I have over a terabyte of photos and videos I've taken). It's encrypted, but I've written the key down on paper and put it in my physical documents. If something happens to me, someone who goes through my physical documents will have access to my digital files.
I pay a cloud service (Backblaze) for cloud backups. I trust the encryption and key management to not actually give the service provider any access to my files.
Thanks!
Just one question: How do you sync your photos to your NAS on the go?
I sync if I have a good Internet connection, like from my hotel room or whatever, by VPNing into my home network where my NAS is. There are distributed DNS type solutions for a lot of the big NAS brands, where they'll let you access your data through their service, but I never set that up because I already have a VPN. So my NAS and firewall are configured not to allow outside connections to that device.
But if I haven't synced laptop to NAS yet, then copies exist on both my camera SD cards (redundant double SD card) and my laptop.
Ah, alright
I currently use wireguard to access my home LAN. just hoped you had done experience with a nice system, so you don't need to do it by hand - especially syncing my smartphone photos.
Maybe I'll give immich another chance
Thanks for the answers! :-)
Fair point. Data integrity and availability concerns often trump confidentiality.
Still, if your main concern is safe disposal rather than preventing unauthorized access, most major operating systems have ways to ensure continued access.
I’ll take a shot at the metaphor. Delete: telling someone they can use the whiteboard (but you didn’t erase it) Secure: cleaning it off when you’re done for the next user.
Computers use an address to tell it where the data is stored on your memory drive
Erase just deleted the address referencing the location, meaning other things can overwrite it.
Secure erase actually deletes the address and overwrites the data that was at where the address was pointing to. Sometimes it’s all zeros, some programs will write random crap over it
Where do people see a secure erase option?
Less of an option and more of choosing the right tool to do the trick.
In day-to-day life I might not care how files are deleted. But when I intend to sell my drive on ebay or so I would make sure that really all the data ia gone.
But how do you choose secure erase? A program or what
Yes.
There’s likely usb-keys to boot
This is an option in the BIOS (UEFI) of certain PCs. I can confirm it exists on a number of HP Elitebook models.
There's a map that shows where every file on your disk is. Doing a regular trash just erases the map pointer for that file, but the 1s and 0s that the file is made of are still sitting there on your disk. Secure erase writes 0s into that area on the disk, so even if you knew where that file used to be located on the disk, now you'll just find 0s there, instead of finding the 1s and 0s of the old file.