SD-card filesystem benchmarks

Prologue: Partition alignment

If you care about your flash chips, reliability and performance you should make sure that your partitions are align to erase block boundaries. Check the flashbench utility if you are unsure.

Update: It seems some cards don't show any performance degradation regardless of partition alignment. The third card I tested (SanDisk Ultra 16GB microSDHC) and a card of OmegaPhil from the openpandora boards (also a SanDisk Ultra SDHC card) are examples of this.

Tested filesystems

ext2

ext2 is often promoted as THE filesystem for use on flash devices. The main reasoning is that it has no journal and thus makes fewer writes to the flash chips. As the number of writes to any flash chip is limited this should increase the chips life expectancy. If this really does matter with modern flash chips and reasonable wear leveling is a matter of personal preference.

ext3

ext3 is the successor to ext2. The main difference to ext2 is that it added a journal. If this is a good or bad thing on a flash chip is up to you to decide.

ext4

ext4 has a couple of new features like extends that are designed to increase filesystem performance. It also has the ability to be used with or without a journal. So it is a candidate for both the "Journal on flash is BAD"- and the "I don't care if my card dies in 50 or 25 years"-camps. Just specify "-O ^has_journal" when creating the filesystem to disable the journal.

Unfortunately on the Pandora ext4 is compiled as a module. So it is not available at system boot time and presently cannot be used for your root filesystem. Das U-Boot on the Pandora also has no ext4 support although this could be circumvented with a small dedicated /boot-partition once the kernel includes ext4 support.

Update: Ext4 is compiled into the kernel as of the final SuperZaxxon firmware. So it is possible to use it as a root filesystem if you use a kernel from NAND or an ext2/3/vfat partition.

As ext4 is the latest incarnation of the ext filesystem family it does not have as much support from within Windows. Wikipedia references a tool Ext2Fsd that can read and write ext4 from Windows. As I have no experience with this utility, you better test it out yourself should you need to write to your card from Windows.

vfat

Everybody knows vfat. It has been around since the stone age. While it lacks modern features (or any features at all for that matter :-)) it has widespread support and anything that can access your SD-Card will be able to read it. Some SD-cards may even have an optimized region at the beginning where the FAT is stored. So it might have the home advantage. But don't even think about putting your root filesystem on a vfat partition unless you want your kernel to laugh at you.

Mount options

"atime" or access time is a feature that records the time a file (or directory) was last accessed. While this may sound nice this adds a write to every read you do on your card. (Read this LWN article if you want to know more.)

The most widely spread "fix" to the access time performance problem is the "noatime" mount option. This disables recording the atime completely. Unfortunately this might break some appliciations (like some mail readers) that need to know if a file was accessed after it was written. That's where the "relatime" options comes in. It only makes atime updates if the file was modified since the last access. The overhead of this is minimal but it should fix most (of the few) applications that have a problem with "noatime".

As the standard mount options on the Pandora include "dirsync" this was also benchmarked. When the option "dirsync" is specified changes to directory entries are written to the disk at once. This is pretty bad for performance but reduces the chance of filesystem corruption if you are intend on yanking out the SD-card without unmounting it first. If you always unmount your media first you can and should remove this option.

Extended options for ext3/4

Ext3 and ext4 also support some options that are designed to increase performance on RAID systems. These are called "stride" and "stripe-width". They modify the way metadata is placed on the disk. To check if it impacts the performance if the metadata is placed differently on a SD-card I tried a lot of different combinations. The tests ran for over 20 hours but I did not find any indication that using the options increases performance.

Testing

Bonnie++ installed on a 512MB Pandora in a Debian squeeze chroot with schroot was used as the benchmarking program. After filesystem creation the first run of Bonnie++ was discarded. Then the filesystem was mounted with "relatime", "noatime" and then "noatime,dirsync" in turn. Each time five Bonnie++ runs were made. Before each run a sync was issued and a wait for one minute to clear anything that the SD-card controller might still had to do. The results are the averages of all five runs. On thing I noticed is that the results were very variable even with the same filesystem and mount options. So you should ignore differences under 10%.

I used my Pandora with a Platinum 64 GB SDXC Class 10 card for testing. Using dd on blockdevice level it can read and write at about 15 MB/s. It has an erase block size of 8MB and can handle 5 open allocation units.

The second card I used was an extremememory Performance 16GB SDHC Class 6 card. Using dd on blockdevice level it can read at about 13.5 MB/s and write at 16.1 MB/s (yes, it actually writes faster than it reads). It has an erase block size of 2MB and can handle 3 open allocation units. All tests on this card were performed 6 times (all with "relatime") and the results of the first test run were discarded again.

The third card was a SanDisk Mobile Ultra 16GB microSDHC Class 10. Using dd on blockdevice level it can read at about 18.8 MB/s and write at 12.6 MB/s. It has an erase block size of 4MB and can handle 5 open allocation units. Testing procedures were the same as for the second card.

You can also download the raw Bonnie++ results (64GB SDXC: CSV, HTML, 16GB SDHC: CSV, HTML, 16GB microSDHC CSV, HTML) if you want to look at the whole story.

Conclusion

In short: Use ext4. That's it. It doesn't matter if you don't want a journal on your flash card. In that case just use ext4 without one.

Slightly longer version: While journaling does slow file creation, ext4 is far superior to the other filesystems tested. Either the results were similar or ext4 was far ahead. It doesn't matter if you don't want a journal on your flash card. Just use ext4 without one (mkfs.ext4 -O ^has_journal /dev/yourblockdevice). Even with a journal ext4 leaves ext2 far behind in a couple of tests.

And if you always unmount your filesystems before removing the card (which is a good idea anyway) you should not use "dirsync" as it cripples performance on file creation and deletion. And while you are at it you can also replace the "noatime" option with "relatime" as the performance penalty is neglectable but it might help prevent problems with a few applications.

If you want to boot your Pandora from an SD-card, ext4 is not an option at the moment unfortunately. So just pick ext3 if you want journaling or ext2 if you don't.

And again: Don't forget to align your partitions to the erase block boundaries. If you don't, this will kill the performance of metadata updates regardless of which filesystem you use. Update: Unless you have a card that is somehow not affected by this. (Although I would still recommend it to be on the "safe side" as I guess the extra write cycle is still necessary on these cards even if it does not have a measurable performance impact.)

vfat Performance

vfat did not perform too shabby on bulk I/O and even was the clear winner in the category "Random seeks" on the 64GB SDXC card although it lost it's edge when handeling a lot of files. What really suprised me though was that it was actually faster with "dirsync" on reading file metadata (at least on sequentially created files). How and why this is possible I have absolutely no idea.

So if you just need to "put something on the card", don't need any permissions, extended attributes or journals and work a lot with Windows, vfat might not be such a bad choice.

Results

Sequential output

Char output

This was skipped as I do not see the relevance of this test and did not want to wait several weeks for the test to finish.

Block output

In this test a file is created and filled with 1G of data written in blocks (not single characters as in the test above).

Rewrite

In this test the file created in the last test is written to again, replacing all the original content with 1 GB of new data.

Sequential input

Char input

Test skipped.

Block input

The file that was written in the last test, is read from start to end.

Random seeks

8.000 seeks within the 1 GB-file. In 10% of the seeks the read block is overwritten with new data.

Sequential create

Create

During this test 102.400 empty files are created in 100 directories (10.240 files in 10 directories for the "dirsync"-test) in alphabetical order.

Read

The properties of each directory entry is read in the order the filesystem stored them in.

As "only" 10.240 files were created with the "dirsync" mount option, the test performed very fast. "About 20.000 /s" means that some tests runs finished in less than 0.5 seconds and Bonnie++ did not give a result for these. Others finished in more than 0.5 seconds and Bonnie++ printed a result of about 20.000 /s.

Delete

And here they are deleted again in the same order.

Random create

Create

Again 102.400 (10.240) empty files are created. But this time the filenames are randomized.

Read

102.400 (10.240) random file properties are read. (This does not have to be and probably is not every file once but some are read several times.)

As "only" 10.240 files were created with the "dirsync" mount option, the test performed very fast. "About 20.000 /s" means that some tests runs finished in less than 0.5 seconds and Bonnie++ did not give a result for these. Others finished in more than 0.5 seconds and Bonnie++ printed a result of about 20.000 /s. "> 20.000 /s" means that all test runs finished in less than 0.5 seconds.

Delete

The files are deleted in random order.