this post was submitted on 06 May 2024
64 points (97.1% liked)
Linux Gaming
15226 readers
870 users here now
Discussions and news about gaming on the GNU/Linux family of operating systems (including the Steam Deck). Potentially a $HOME
away from home for disgruntled /r/linux_gaming denizens of the redditarian demesne.
This page can be subscribed to via RSS.
Original /r/linux_gaming pengwing by uoou.
Resources
WWW:
Discord:
IRC:
Matrix:
Telegram:
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Btrfs is amazing for a steam library. The single best feature is the compression. Games tend to have lot of unoptimized assets which compress really well. Because decompression is typically faster than your disk, it can potentially make games load faster too.
I put a second dedicated nvme drive in my PC just for steam. It's only 512GB but it holds a surprisingly large library.
and ~~my axe~~ deduping. all those dlls and wine prefixes that contain them occupy space only once.
I actually found the opposite with my steam library; on ZFS with ZSTD I only saw a ratio of 1.1 for steamapps, not that there's really any meaningful performance penalty for compressing it.
It depends on what sort of games you play. Some games / genres / publishers are much worse about this than others.
OK I just measured mine. I have 459GiB of games on the drive, consuming 368GiB of space. That's about 25% compression. I'm using
compress=zstd:9
.I should try deduplication. I have 4 steam users and I've created an ACL hell to prevent the same game being downloaded and installed twice.
If you're messing with ACLs I'm not sure deduplication will help you much; I believe (not much experience with reflinks) the dedup checksum will include the metadata, so changing ACLs might ruin any benefit. Even if you don't change the ACLs, as soon as somebody updates a game, it's checksum will change and won't converge back when everyone else updates.
Even hardlinks preserve the ACL... Maybe symlinks to the folder containing the game's data, then the symlinks could have different ACLs?
I wrote a blog about it last year with my method of deduplicating. I really need to update that bit because steam keeps writing files that don't uphold the group permissions, and others get permission errors that need to be fixed by admin. Steam also failed to determine free space on a drive when symlinks were involved.
I even found recently that steam would write files in /tmp/ as one user, and fail when you logged in as another user and tried to write the same file. Multi-user breaks even without messing around.
My current solution doesn't use symlinks. I just add two libraries for each user. One in their respective home directory, and another shared in /mnt/steam. It means that any user can update a game in /mnt/steam, and it cleanly updates for all users at once.
Is the compression opt-in or is it enabled by default?
You have to enable compression in fstab.
Ah okay, cool. It's that easy? Does it compress all existing data after that or is it only for new data?
What would I have to do to compress existing data?
It is only for new data.
For example, you would have to defragment your filesystem again with
btrfs filesystem defragment -r -v -czstd /
. Wherezstd
is an algorithm and/
, a root path. With this command, the default compression level will be used, which is level 3.Be careful, defragmenting the btrfs file system will/can duplicate the data.
As for a mount point, if you decided to use zstd algorithm with level 1 compression, just add the
compress=zstd:1
orcompress-force=zstd:1
to the mount options (fstab or while mounting manually)Reading the manpage (btrfs-filesystem), duplication can happen on some odd kernel versions, so no danger.
Edit: that was my interpretation of breaking up reflinks of cow data anyway. Seems there's more.
If I know correctly, defrag will always duplicate the reflink files.
https://btrfs.readthedocs.io/en/latest/Defragmentation.html
Well, compression doubled my available space. ;-)
So I set up my system with btrfs in the last days and I converted two external drives (from ext4) (mainly game) and run defrag and balance, because it was mentioned in a guide to compress the existing files. Was that a bad idea? Didn't read anything about duplicates.
It is fine. You can use the duperemove tool (or bees) to find and remove duplicates.
https://btrfs.readthedocs.io/en/latest/Deduplication.html
So it is out-of-band deduplication and has to be done manually.
Also, by default cp and most file managers use a reflink copy (data blocks are copied only when modified)