Fstrim, Lvm and Dmcrypt

Blog Projects About me Contact

Tweet synaptic fault mind dumped

How to properly activate TRIM for your SSD on Linux: fstrim, lvm and dmcrypt Language

Posted on Saturday, 23rd February 2013 27 comments English Español Unlike hard disk drives (HDDs), NAND flash memory that make SSD cannot overwrite existing data. This means that you first have to delete the old data before writing new one. Flash memory is divided into blocks, which is further divided in pages. The minimum write unit is a page, but the smallest erase unit is a Tags block.

apache Applet debian email fpga game Git GNOME

hack hardware kernel

linux meego networking packages performance power management procmail python RFKill Security ssd tor trim usb virtualization war webkit wireless

Categories

http://planet.igalia.com http://planet.webkit.org http://planet.webkitgtk.org http://planetsecurity.org

Share it!

Data can be written directly into an empty page, but only whole blocks can be erased. Therefore, to reclaim the space taken up by invalid data, all the valid data from one block must be first copied and written into the empty pages of a new block. Only then can the invalid data in the original block be erased, making it ready Recent Posts for new valid data to be written. WebKitGTK+ Performance Bot! Do you see the problem? This means that as time goes on, the SSD will internally fragment the blocks How to connect to .onion Tor domains with among the different pages, until that it reaches a point where there won’t be available any empty page. Firefox on Linux Then every time the drive needs to write a block into any of the semifull pages, it first needs to copy the How to properly activate TRIM for your SSD on current blocks from the page to a buffer, then it has to delete the whole page to finally rewrite the old blocks along with the new one. This means that as time goes on the SSD performance degrades more and more, Linux: fstrim, lvm and dmcrypt because for every write it has to go through a cycle of readerasemodifywrite. This is known as “write Git: Copy a file or directory from another amplification”. repository preserving the history IKEA Hackers: LackRack Without TRIM the disk is unable to know which blocks are in use by a file or which ones are marked as free Unlocking a LUKS encrypted root partition space. This is because when a file is deleted, the only thing the OS does is to mark the blocks that were remotely via SSH used by the file as free inside the file system index. But the OS won’t tell the disk about this. This means OpenRISC: Making the stack 100% free that over time the performance of the SSD disk will degrade more and more, and it don’t matters how much Don’t eject your USB storage: stop it! space you free, because the SSD won’t know about it. RFKill Applet for GNOME What is TRIM? TRIM was invented for solving this problem. TRIM is the name of a command that the Auth External: the Swiss Army Knife of Apache operating system can send to tell the SSD which blocks are free in the filesystem. Auth The SSD uses this information to internally defragment the blocks and keep free pages available to be written quickly and efficiently. Tweets

How to active TRIM on Linux? The first thing to know is that TRIM should be enabled on all I/O abstraction RT @News_Executive: Update: layers. This means that if you have an ext4 partition on top of LVM, which in turn is on top of an encrypted Spectacular Photo of the new eruption volume with LUKS/dmcrypt, then you must enable support for TRIM in these three layers: The filesystem, now of #VolcanCalbuco in #Chile LVM and dmcrypt. There is no point in enabling it at the filesystem level if you don’t enable it also on the http://t.co/JcXBQ9lybC other layers. The TRIM command should be translated from one layer to another until reaching the SSD. 1 month ago

RT @aszy: Improving Compiler Optimizations Using Machine Learning http://t.co/jfVOO1kfSG 1 month ago

Microsoft is going to celebrate the next Debian release!. Not a joke: http://t.co/1NO3dZCjaR @debian 1 month ago

RT @r_netsec: Practical Attacks on DOCSIS [PDF] http://t.co/jzYjnytQZa 1 month ago

RT @GSICKMINDS: This year to the #gsickminds2015 we have adopted our friends @jsconf code of conduct http://t.co/52U2KdAngu 2 months ago

1. Enabling TRIM support on dmcrypt

We simply have to add the option discard inside our crypttab

$ cat /etc/crypttab

# sda2_crypt /dev/sda2 none luks,discard

Note: The usage of TRIM on dmcrypt could cause some security issues like the revelation of which sectors of your disk are unused.

2. Enabling TRIM support on LVM

We have to enable the option issue_discards in the LVM configuration.

$ cat /etc/lvm/lvm.conf

# [...] devices { # [...] issue_discards = 1 # [...] } # [...]

3. Enabling TRIM support on the file system

This is the most interesting part. Most people simply add the option “discard” in the mounting options at /etc/fstab. However, this means that every time you delete a file, the OS will be reporting in realtime to the SSD which blocks were occupied by that file and are not longer in use, and then the SSD will have to perform a defragmentation and deletion of those internal blocks, operation which will take an amount of time higher than desired.

In order to optimize the performance of the SSD, I strongly advise you to avoid doing the TRIM operation in real time (whenever a file is deleted) because you would be putting an unnecessary extra amount of work over the SSD. In other words: You should not enable the discard option in fstab.

Instead, what I recommend is to run a script periodically to tell the SSD which blocks are free with the command fstrim. Doing this operation daily or weekly is more than enough. This way we do not lose any performance due to TRIM when deleting files and we periodically keep informed the SSD about the free blocks.

Other advantages of the fstrim way are:

If you didn’t enabled correctly the TRIM support in the above layers of your setup, you will receive an error when executing fstrim. On the other hand, if you were using the discard option at fstab you wouldn’t have received any error and you would end thinking that you managed to get TRIM working properly when you didn’t. If you delete a file by mistake (you know it happens), you can recover it before anacron runs your script fstrim. On the other hand, if you were using the discardatfstab option you wouldn’t have any chance of recovering the file, because the OS would have told the SSD to TRIM that blocks as soon as you deleted the file, and consequently the SSD has irreversibly destroyed such blocks.

Here you have simple script to run fstrim on the /, /boot and /home partitions, which can be programmed to be executed periodically by anacron

$ cat /etc/cron.weekly/dofstrim

#! /bin/sh for mount in / /boot /home; do fstrim $mount done

It’s worth mentioning that both fstrim and discardatfstab options only work on filesystems that implement the ioctl FITRIM. Today, such filesystems are:

~/kernel/linux (v3.8) $ grep lr FITRIM fs/ | cut d/ f2 | sort | uniq | xargs echo btrfs ext3 ext4 gfs2 jfs ocfs2 xfs

Note: On most setups you will have to rebuild your initramfs with updateinitramfs u (Debian and derivatives) or dracut f (Redhat and derivatives) and reboot the machine after touching the configuration options of LVM or dmcrypt.

Update 25Feb2013: Seems that Fedora users with a dmcrypt volume will be affected by this problem: https://bugzilla.redhat.com/890533

Update 14Aug2014: The following script can be used to automatically detect and fstrim all filesystems that have TRIM support enabled.

# To find which FS support trim, we check that DISCMAX (discard max bytes) # is great than zero. Check discard_max_bytes documentation at # https://www.kernel.org/doc/Documentation/block/queuesysfs.txt for fs in $(lsblk o MOUNTPOINT,DISCMAX,FSTYPE | grep E '^/.* [19]+.* ' | awk '{print $1}'); do

linux, ssd, trim

← Git: Copy a file or directory from another repository How to connect to .onion Tor domains with Firefox on preserving the history Linux →

Leave a comment ? 27 Comments.

charlie Friday, 12th April 2013 at 15:50 UTC Reply the default install with encryption sets /boot as ext2. and i guess that doesn’t support Trim. what will the ramifications be to leave /boot as ext2 and not be able to Trim?

Reply clopez Friday, 12th April 2013 at 19:14 UTC

/boot is usually a very small partition (<500MB usually). So I think is not a big deal to have this without trim support. If you are worried by this, you can change the default to use ext3 for /boot

Reply Flaburgan Thursday, 31st October 2013 at 7:04 UTC

What about the swap partition? Should I run fstrim on it too?

charlie Friday, 12th April 2013 at 21:37 UTC Reply thank you! excellent blog you have!

Exon Thursday, 9th May 2013 at 15:30 UTC Reply Thanks for your great information. I used to add discard in fstab before. When I switched to LVM, I needed to implement another way to set the Trim in my SSD. I have followed the blog information to setup the Trim, is there any method to check my setup is correct? Just like https://sites.google.com/site/lightrush/random1/checkiftrimonext4isenabledandworking or http://howto.unixdev.net/Test_LVM_Trim_Ext4.html

Reply clopez Thursday, 9th May 2013 at 16:22 UTC If you don’t get any error when you run fstrim then all should be working correctly. fstrim will give you an error if TRIM support is not enabled correctly

Reply Exon Saturday, 11th May 2013 at 2:27 UTC

Thanks clopez!

Christian Tuesday, 4th June 2013 at 23:48 UTC Reply Excellent article thanks!

pcworld Monday, 5th August 2013 at 1:51 UTC Reply Is this still true? (from http://askubuntu.com/a/191945/178920)

However, these ‘discards’ will only trigger on remove or reduce operations on LVs. Discards on the filesystem on top of a LV will not propagate! From the same LVM changelog: “Issue discards on lvremove and lvreduce etc. if enabled and supported.”

Reply clopez Monday, 5th August 2013 at 2:36 UTC

Yes. Not sure about Ubuntu 12.04, but at least on Debian 6 (Squeeze) is true. The support for TRIM to LVM was added in the version 2.02.85 (see changelog) So, if your Linux distribution ships an older version of LVM you may get no support for TRIM.

Reply pcworld Monday, 5th August 2013 at 14:10 UTC

But doesn’t that mean the “discard” option in fstab is useless? Or does fstrim work around LVM2?

Reply clopez Monday, 5th August 2013 at 16:24 UTC

If your filesystem is on top of a LVM volume, and your version of LVM is older than 2.02.85 you won’t get TRIM support. Neither with the discard option nor the fstrim one. The good thing about fstrim is that it will report the errors. However discard option in fstab will fail silently.. So the best advice to check if your FS has TRIM support enabled on all the required layers is to run fstrim. If fstrim works, then you have TRIM enabled correctly, otherwise you will get an error.

John Wednesday, 21st August 2013 at 12:12 UTC Reply Not being much of a bash person a couple of things confuse me on this. My machine is set up with system files on SSD and /tmp /var and /home on hard drives to avoid writes to the ssd. This leaves me wondering what to fstrim as all of these come off /. The other aspect comes from reading about using ssd disks in redhat docs. They suggest trimming when the ssd has run out of “fresh space”. Makes sense as this would ensure wear levelling. There doesn’t seem to be a command available to check how much “fresh space” is left though. All not helped by the fact that there seems to be little info about on what exactly goes on inside these disks.

Thanks for the info by the way. Best I have found. John

consatan Monday, 28th October 2013 at 9:10 UTC Reply How can I fstrim swap partition?

Reply clopez Monday, 28th October 2013 at 11:07 UTC

Discard is a valid mount option (/etc/fstab) for swap. The other option is to use the flag d when adding the swap. Check the manpage of swapon

Shash Saturday, 21st December 2013 at 21:09 UTC Reply Hi Clopez,

Thanks for sharing this info – Gracias I reached this page, by going through your original question on Serverfault.com (

with ref. to your question – you had asked about doing this setup w/ RAID 0: —FS (OS)—| ——LVM——| —RAID 1—–| —dmcrypt—| —SSD1+SSD2—v

However, your post here doesn’t cover the RAID 0 setup part. Highly appreciate if you can share on how software RAID 1 was achieved (if at all) in this setup. Many thanks. cheers, s#

Reply clopez Saturday, 1st February 2014 at 21:35 UTC

This question on serverfault.com is not from myself.

About TRIM over RAID, I never tested it, but my understanding is that it should work out of the box if you have a recent enough kernel that supports it. MD RAID needs specific support to forward DISCARD requests and that hasn’t even been proposed until 3.6 https://lkml.org/lkml/2012/3/11/261

Reply shash Wednesday, 19th February 2014 at 12:31 UTC

Hi Clopez,

thanks for tip. did realize that I have got the RAID #s all over the place in my original comment :D…. but I guess you did get the drift that I was looking for RAID 1 :). cheers. S#

oxtan Monday, 6th January 2014 at 18:50 UTC Reply I have this bash snippet for the cron.weekly job that gets the mountpoints based on the mounted devices info instead of hard coding them. Maybe it is useful to somebody else:

#!/bin/sh

# this cronjob will discard unused blocks on the ssd mounted filesystems # get the locally mounted block devices those starting with "/dev: # run df k, pipe the result through grep and save the sixth field in # in the mountpoint array mountpoint=( $(df k | grep ^/dev | awk '{print $6}') )

# loop through the array and run fstrim on every mountpoint for i in "${mountpoint[@]}" do /usr/sbin/fstrim v $i; done

ryno75 Monday, 27th January 2014 at 23:05 UTC Reply Fantastic writeup! Really well written and organized. One of the most useful howto’s I’ve seen. ever.

Frank Wang Sunday, 9th March 2014 at 11:43 UTC Reply Great blog! The only problem I encountered in my Ubuntu 12.04 where / is on lvm is the “fstrim /” command can still be run without any error raised even if I added ‘issue_discard = 0′ in lvm.conf, then ‘updateinitramfs u’ and reboot performed. This is not expected. Any comment? Thanks!

Ralf Thursday, 17th April 2014 at 15:06 UTC Reply It seems that openSUSE 13.1 is currently not prepared for proper handling of the “discard” option in /etc/crypttab. The initrd does not contain the proper command.

Workaround: In order to activate discard on boot time, I just hardcoded the “cryptsetup” commands in /lib/mkinitrd/boot/71luks.sh to contain “luksOpen –allowdiscards”. Works fine so far.

mr. Coconut Wednesday, 4th June 2014 at 17:58 UTC Reply Hi, it seems that trim is now activated by default on ubuntu 14.04 lts and its derivatives. However, if you are using LVM (not encrypted) and have snapshots of logical volumes, then the system is unable to trim. I can confirm this behaviour. Any ideas?

XTL Saturday, 5th July 2014 at 13:31 UTC Reply On Debian (and the like) you can also do

#! /bin/sh for mount in / /boot /home; do fstrim $mount done

Reply XTL Saturday, 5th July 2014 at 13:32 UTC

Isn’t that a nice paste slip No preview here. $ sudo dpkgreconfigure initramfstools