I recently decided to reinstall my system and at the same time try a new file system. Trying to decide what filesystem to use I found a few benchmarks but either they don't compare all available fs's, are too synthetic (copy a source tree multiple times or raw i/o), or are meant for servers/databases (like
Bonnie++∞). The two most file system intensive tasks I do regularly are `apt-get upgrade` waiting for the packages to extract and set themselves up and messing around with the kernel so I benchmarked these. To make it more realistic I installed
ccache∞ and did two compiles, one to fill the cache and a second using the full cache.
The Tests
The tests I timed (in order)
- Debootstrap to install base debian system
- Extract the kernel source
- Run `make all` using the defconfig and an empty ccache
- Copy the entire new directory tree
- Run `make clean`
- Run `make all` again, this time using the filled ccache
- Deleting the entire directory tree
Summary
Below is a summary of the results based upon what I am calling "dead" time calculated as `total time - user time`. As you can see in the full results below the user time is almost identical between filesystems, so I believe this is an accurate comparison. The dead time is then normalized using ext2 as a baseline (> 1 means it took that many times longer than ext2).
| Filesystem |
deb |
tar |
make |
cp |
clean |
make2 |
rm |
total |
| ext2 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
| ext3 |
1.12 |
2.47 |
0.88 |
1.16 |
0.91 |
0.93 |
3.01 |
1.13 |
| jfs |
1.64 |
2.18 |
1.22 |
1.90 |
1.60 |
1.19 |
12.84 |
1.79 |
| reiser |
1.12 |
1.99 |
1.05 |
1.41 |
0.92 |
1.56 |
1.42 |
1.28 |
| reiser4 |
2.69 |
1.87 |
1.80 |
0.63 |
1.33 |
2.71 |
4.14 |
1.83 |
| xfs |
1.06 |
1.99 |
0.97 |
1.67 |
0.78 |
1.03 |
10.27 |
1.43 |
Comments
- Ext2 is still overall the fastest but I think the margin is small enough that a journal is well worth it
- Ext3, ReiserFS, and XFS all perform similarly and almost up to ext2 except:
- XFS takes an abnormally long time to do a large rm even though it is very fast at a kernel `make clean`
- ReiserFS is significantly slower at the second make (from ccache)
- JFS is fairly slow overall
- Reiser4 is exceptionally fast at synthetic benchmarks like copying the system and untaring, but is very slow at the real-world debootstrap and kernel compiles.
- Though I didn't benchmark it, ReiserFS sometimes takes a second or two to mount and Reiser4 sometimes takes a second or two to unmount while all other filesystem's are instantaneous.
Originally I had planned on using Reiser4 because of the glowing reviews they give themselves but I'm simply not seeing it. It might be that my Reiser4 is somehow broken but I don't think so. Based on these results I personally am now going with XFS as it's faster than ReiserFS in the real-world benchmarks and my current Ext3 partition's performance is getting worse and worse.
Program
I hacked together a simple shell script to automate these benchmarks for me. All it depends on is having
debootstrap∞ installed (any Debian system has it or you can try to deb-to-rpm it or something). It's probably fairly unclean with the forced `mkfs` and `rm -rf` so after you change the configuration variables at the top, please read through and understand the source (it's all commented) before you accidently reformat your master partition. You might also have to change EXTRAS as dependencies change.
fsbench.sh∞
System
Here are the specs of the desktop these tests were run on:
- Kernel: Vanilla 2.6.3 with reiser4 2004.02.06 patch
- Distro: Debian Unstable
- CPU: Athlon XP 2800+
- RAM: 1GB DDR 333
- Motherboard: VIA KT400
- Hard Drive: WDC WD800JB (80gb, 8mb cache)
- Partition: 3GB partition starting from the 8GB mark
Program Versions
- mke2fs 1.35-WIP (31-Jan-2004)
- mkfs.jfs version 1.1.4, 30-Oct-2003
- mkfs.reiser4 0.5.0
- mkfs.reiserfs 3.6.13
- mkfs.xfs version 2.6.3
Full Results:
Here are the complete results for each test in both real time and normalized to ext2. Each table is sorted by real execution time.
debootstrap
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext2 |
24.38 |
1.00 |
11.87 |
1.00 |
3.77 |
1.00 |
28.54 |
1.00 |
52.92 |
1.00 |
| xfs |
25.13 |
1.03 |
14.90 |
1.26 |
3.74 |
0.99 |
30.36 |
1.06 |
55.49 |
1.05 |
| ext3 |
24.73 |
1.01 |
12.62 |
1.06 |
0.28 |
0.07 |
31.91 |
1.12 |
56.64 |
1.07 |
| reiser |
25.08 |
1.03 |
16.23 |
1.37 |
2.54 |
0.67 |
31.99 |
1.12 |
57.07 |
1.08 |
| jfs |
24.84 |
1.02 |
12.77 |
1.08 |
5.60 |
1.49 |
46.78 |
1.64 |
71.62 |
1.35 |
| reiser4 |
25.37 |
1.04 |
20.78 |
1.75 |
0.08 |
0.02 |
76.73 |
2.69 |
102.10 |
1.93 |
tar jxf
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext2 |
21.94 |
1.00 |
1.51 |
1.00 |
2.98 |
1.00 |
5.29 |
1.00 |
27.23 |
1.00 |
| reiser4 |
22.51 |
1.03 |
4.07 |
2.70 |
2.55 |
0.86 |
9.88 |
1.87 |
32.39 |
1.19 |
| reiser |
22.23 |
1.01 |
3.37 |
2.23 |
5.67 |
1.90 |
10.52 |
1.99 |
32.75 |
1.20 |
| xfs |
22.35 |
1.02 |
3.10 |
2.05 |
3.27 |
1.10 |
10.54 |
1.99 |
32.89 |
1.21 |
| jfs |
22.00 |
1.00 |
2.17 |
1.44 |
4.92 |
1.65 |
11.53 |
2.18 |
33.53 |
1.23 |
| ext3 |
21.88 |
1.00 |
2.21 |
1.46 |
6.37 |
2.14 |
13.09 |
2.47 |
34.97 |
1.28 |
make all (after make defconfig)
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext3 |
301.09 |
1.00 |
27.69 |
1.03 |
0.98 |
0.53 |
29.85 |
0.88 |
330.94 |
0.99 |
| xfs |
301.39 |
1.00 |
29.31 |
1.09 |
1.58 |
0.85 |
32.68 |
0.97 |
334.07 |
1.00 |
| ext2 |
301.80 |
1.00 |
26.99 |
1.00 |
1.85 |
1.00 |
33.77 |
1.00 |
335.57 |
1.00 |
| reiser |
301.40 |
1.00 |
30.93 |
1.15 |
1.62 |
0.88 |
35.61 |
1.05 |
337.01 |
1.00 |
| jfs |
301.07 |
1.00 |
27.30 |
1.01 |
2.76 |
1.49 |
41.05 |
1.22 |
342.12 |
1.02 |
| reiser4 |
302.63 |
1.00 |
57.46 |
2.13 |
2.47 |
1.34 |
60.73 |
1.80 |
363.36 |
1.08 |
cp -a $MNT/* $MNT/z
| Filesystem |
usr |
sys |
sync |
dead |
real |
| reiser4 |
0.31 |
1.19 |
6.45 |
1.84 |
5.97 |
0.70 |
23.35 |
0.63 |
23.66 |
0.63 |
| ext2 |
0.26 |
1.00 |
3.50 |
1.00 |
8.53 |
1.00 |
37.09 |
1.00 |
37.35 |
1.00 |
| ext3 |
0.27 |
1.04 |
4.78 |
1.37 |
18.97 |
2.22 |
42.96 |
1.16 |
43.23 |
1.16 |
| reiser |
0.29 |
1.12 |
6.21 |
1.77 |
7.98 |
0.94 |
52.28 |
1.41 |
52.57 |
1.41 |
| xfs |
0.36 |
1.38 |
7.30 |
2.09 |
2.86 |
0.34 |
61.88 |
1.67 |
62.24 |
1.67 |
| jfs |
0.23 |
0.88 |
4.16 |
1.19 |
4.35 |
0.51 |
70.53 |
1.90 |
70.76 |
1.89 |
make clean (in kernel)
| Filesystem |
usr |
sys |
sync |
dead |
real |
| xfs |
1.13 |
1.02 |
0.62 |
1.41 |
0.67 |
1.08 |
2.77 |
0.78 |
3.90 |
0.84 |
| ext3 |
1.12 |
1.01 |
0.49 |
1.11 |
1.38 |
2.23 |
3.20 |
0.91 |
4.32 |
0.93 |
| reiser |
1.12 |
1.01 |
0.59 |
1.34 |
1.27 |
2.05 |
3.24 |
0.92 |
4.36 |
0.94 |
| ext2 |
1.11 |
1.00 |
0.44 |
1.00 |
0.62 |
1.00 |
3.53 |
1.00 |
4.64 |
1.00 |
| reiser4 |
1.11 |
1.00 |
0.79 |
1.80 |
2.72 |
4.39 |
4.69 |
1.33 |
5.80 |
1.25 |
| jfs |
1.12 |
1.01 |
0.49 |
1.11 |
1.29 |
2.08 |
5.65 |
1.60 |
6.77 |
1.46 |
make all (in kernel with full ccache)
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext3 |
41.88 |
1.00 |
12.80 |
1.09 |
1.40 |
0.44 |
16.85 |
0.93 |
58.73 |
0.98 |
| ext2 |
41.71 |
1.00 |
11.71 |
1.00 |
3.21 |
1.00 |
18.08 |
1.00 |
59.79 |
1.00 |
| xfs |
42.52 |
1.02 |
13.60 |
1.16 |
2.54 |
0.79 |
18.66 |
1.03 |
61.18 |
1.02 |
| jfs |
41.88 |
1.00 |
12.74 |
1.09 |
4.65 |
1.45 |
21.58 |
1.19 |
63.46 |
1.06 |
| reiser |
42.75 |
1.02 |
15.34 |
1.31 |
5.86 |
1.83 |
28.19 |
1.56 |
70.94 |
1.19 |
| reiser4 |
43.37 |
1.04 |
42.37 |
3.62 |
4.06 |
1.26 |
49.02 |
2.71 |
92.39 |
1.55 |
rm -rf $MNT/*
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext2 |
0.01 |
1.00 |
0.67 |
1.00 |
0.28 |
1.00 |
2.67 |
1.00 |
2.68 |
1.00 |
| reiser |
0.04 |
4.00 |
3.39 |
5.06 |
0.00 |
0.00 |
3.80 |
1.42 |
3.84 |
1.43 |
| ext3 |
0.03 |
3.00 |
1.59 |
2.37 |
3.38 |
12.07 |
8.05 |
3.01 |
8.08 |
3.01 |
| reiser4 |
0.07 |
7.00 |
5.87 |
8.76 |
0.41 |
1.46 |
11.06 |
4.14 |
11.13 |
4.15 |
| xfs |
0.08 |
8.00 |
6.13 |
9.15 |
0.03 |
0.11 |
27.42 |
10.27 |
27.50 |
10.26 |
| jfs |
0.04 |
4.00 |
1.38 |
2.06 |
0.04 |
0.14 |
34.29 |
12.84 |
34.33 |
12.81 |
total
| Filesystem |
usr |
sys |
sync |
dead |
real |
| ext2 |
391.21 |
1.00 |
56.69 |
1.00 |
21.24 |
1.00 |
128.97 |
1.00 |
520.18 |
1.00 |
| ext3 |
391.00 |
1.00 |
62.18 |
1.10 |
32.76 |
1.54 |
145.91 |
1.13 |
536.91 |
1.03 |
| reiser |
392.91 |
1.00 |
76.06 |
1.34 |
24.94 |
1.17 |
165.63 |
1.28 |
558.54 |
1.07 |
| xfs |
392.96 |
1.00 |
74.96 |
1.32 |
14.69 |
0.69 |
184.31 |
1.43 |
577.27 |
1.11 |
| jfs |
391.18 |
1.00 |
61.01 |
1.08 |
23.61 |
1.11 |
231.41 |
1.79 |
622.59 |
1.20 |
| reiser4 |
395.37 |
1.01 |
137.79 |
2.43 |
18.26 |
0.86 |
235.46 |
1.83 |
630.83 |
1.21 |
::
Show / Add comments (4)
:: Powered by Wikka Wakka Wiki 1.1.5.2∞