If you are using GELI to encrypt disks on FreeNAS or NAS4Free, there are multiple algorithms you can use. However, in practice, for me this always becomes a choice between AES-XTS and AES-CBC. Since every modern processor had AES primitives supported in hardware, these two offer performance other algorithms cannot reach.
The current industry standard for disk encryption is XTS-AES and that is mode I usually default to. It has been made specifically to facilitate full-disk encryption and has no known attacks (other than malleability attack common to practically all such algorithms).
Using AES-CBC is not as crazy as it seems. While there is a known "watermark" attack, GELI is not vulnerable at it uses CBC-ESSIV. And, while it is still vulnerable against malleability attack, it's more resilient than XTS to some (albeit rather unrealistic) attacks while sharing CBC's "friendliness" to altering bits on per-block level.
In the absence of any new attack, you are probably fractionally more secure with XTS but realistically both XTS-AES and AES-CBC with ESSIV can be considered equivalent and well-suited for full-disk encryption. That said, as security is not concern, are there performance differences between those two?
And there are. At least on NAS4Free albeit I believe the same applies across the BSD family - including FreeNAS.
My ZFS mirror on i3-4010U used XTS-AES-128 for encryption and its median write speed was 197 MB/s. The same mirror had its write speed increased to 389 MB/s once both disks were set to use AES-CBC-128. Interestingly, changing the encryption mode on a single disk to CBC while the other was still using XTS also improved performance - all the way to 384 MB/s. Mind you there are no security benefits running two disks in different encryption modes. I just did it for fun. :)
My Atom C2558 server using software encryption with XTS-AES only wrote at 110 MB/s. Once both physical drives used AES-CBC, speed jumped to 162 MB/s. Again, changing only a single drive in mirror to CBC increased write speed to 157 MB/s.
The same server using hardware encryption could write 160 MB/s when both drives encrypted using XTS-AES. Both drives using AES-CBC achieved again 160 MB/s. Once more, the curiosity was half/half situation with write speed at 162 MB/s.
Based on these results it is obvious that AES-CBC performs much better than XTS-AES in some cases or both have the same speed at worst. The fact changing a single drive to CBC already brought 95% of improvement tells me that CPU was really straining with XTS despite hardware AES-NI support. On the slower CPU difference was non-existent albeit this was more related to its general slowness of AES-NI on Atom than to anything algorithm specific.
Intel's own tests on the same generation (albeit vastly more powerful processors) shows that XTS implementation can actually be faster than CBC - especially when multiple cores can "exploit" XTS' inherent parallelism.
This just shows that performance is highly dependent on everything in the pipeline - especially software support - and not just hardware. And one should be careful to occasionally retest if old assumptions are still valid. For example, the next version of BSD crypto library could improve performance of XTS to surpass CBC.
I personally will still stick with XTS-AES. Encryption and decryption speeds, while not exhilarating, are more than adequate for sharing files over Samba connections. While AES-CBC will surely enjoy support in the future, XTS is the one standardized for full-disk encryption. Unless a security issue is discovered, support for it will only get better with time.
PS: Malleability attack on AES-XTS and AES-CBC is actually not that problematic for my setup as ZFS includes data checksum. To be completely sure, for sensitive applications, one should think about using SHA-256 instead of default Fletcher-16 checksum.
PPS: GELI actually doesn't use ESSIV as you would find it on Linux. However, its implementation is pretty much equivalent to it in both security and performance.
PPPS: For curious, I used the following command to test write speed, repeating the measurement 5 times:
# dd if=/dev/zero of=/mnt/Temp/Test.tmp bs=4096 count=1048576PPPPS: You can also get rough idea which encryption method is faster by running OpenSSL speed test:
# openssl speed -evp aes-128-xts
# openssl speed -evp aes-128-cbc[2018-07-22: NAS4Free has been renamed to XigmaNAS as of July 2018]
Hi.
So, for my test set-up CBC is still way better.
The data is as follows:
XigmaNAS 11.4.0.4 – Accadia (kompilacja 7881) running on AMD Turion(tm) II Neo N54L Dual-Core Processor
128bits
xigmanas: ~# openssl speed -evp aes-128-xts
Doing aes-128-xts for 3s on 16 size blocks: 5940241 aes-128-xts’s in 3.00s
Doing aes-128-xts for 3s on 64 size blocks: 2455985 aes-128-xts’s in 3.05s
Doing aes-128-xts for 3s on 256 size blocks: 733915 aes-128-xts’s in 3.03s
Doing aes-128-xts for 3s on 1024 size blocks: 196914 aes-128-xts’s in 3.09s
Doing aes-128-xts for 3s on 8192 size blocks: 24441 aes-128-xts’s in 3.03s
OpenSSL 1.0.2u-freebsd 20 Dec 2019
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-xts 31681.29k 51588.28k 61981.77k 65176.54k 66052.18k
xigmanas: ~# openssl speed -evp aes-128-cbc
Doing aes-128-cbc for 3s on 16 size blocks: 11547095 aes-128-cbc’s in 3.02s
Doing aes-128-cbc for 3s on 64 size blocks: 3198213 aes-128-cbc’s in 3.01s
Doing aes-128-cbc for 3s on 256 size blocks: 831240 aes-128-cbc’s in 3.02s
Doing aes-128-cbc for 3s on 1024 size blocks: 524029 aes-128-cbc’s in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 67668 aes-128-cbc’s in 3.06s
OpenSSL 1.0.2u-freebsd 20 Dec 2019
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 61107.11k 68051.33k 70382.62k 178868.57k 181007.76k
————————–
256 bits
xigmanas: ~# openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 8383321 aes-256-cbc’s in 2.99s
Doing aes-256-cbc for 3s on 64 size blocks: 2286253 aes-256-cbc’s in 3.03s
Doing aes-256-cbc for 3s on 256 size blocks: 585114 aes-256-cbc’s in 3.02s
Doing aes-256-cbc for 3s on 1024 size blocks: 377998 aes-256-cbc’s in 3.07s
Doing aes-256-cbc for 3s on 8192 size blocks: 47875 aes-256-cbc’s in 3.09s
OpenSSL 1.0.2u-freebsd 20 Dec 2019
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-cbc 44827.78k 48270.58k 49542.68k 126068.58k 127090.07k
xigmanas: ~# openssl speed -evp aes-256-xts
Doing aes-256-xts for 3s on 16 size blocks: 4353417 aes-256-xts’s in 3.02s
Doing aes-256-xts for 3s on 64 size blocks: 1756798 aes-256-xts’s in 3.01s
Doing aes-256-xts for 3s on 256 size blocks: 522111 aes-256-xts’s in 3.00s
Doing aes-256-xts for 3s on 1024 size blocks: 137033 aes-256-xts’s in 3.00s
Doing aes-256-xts for 3s on 8192 size blocks: 17399 aes-256-xts’s in 3.00s
OpenSSL 1.0.2u-freebsd 20 Dec 2019
built on: date not available
options:bn(64,64) rc4(8x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-256-xts 23097.92k 37381.01k 44553.47k 46773.93k 47510.87k
I hope this will help.
Regards,
Marcin
Interesting. It seems AMD likes CBC more :) albeit they’re really close.
Intel seems to like XTS better:
x64-embedded on Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
openssl speed -evp aes-256-xts
Doing aes-256-xts for 3s on 16 size blocks: 14011544 aes-256-xts’s in 3.00s
Doing aes-256-xts for 3s on 64 size blocks: 31793336 aes-256-xts’s in 3.08s
Doing aes-256-xts for 3s on 256 size blocks: 17157441 aes-256-xts’s in 3.00s
Doing aes-256-xts for 3s on 1024 size blocks: 6543325 aes-256-xts’s in 3.06s
Doing aes-256-xts for 3s on 8192 size blocks: 943982 aes-256-xts’s in 3.03s
Doing aes-256-xts for 3s on 16384 size blocks: 476304 aes-256-xts’s in 3.01s
OpenSSL 1.1.1h-freebsd 22 Sep 2020
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-256-xts 74728.23k 661043.17k 1464101.63k 2187874.22k 2551125.95k 2594498.41k
openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 46421523 aes-256-cbc’s in 3.03s
Doing aes-256-cbc for 3s on 64 size blocks: 20135299 aes-256-cbc’s in 3.02s
Doing aes-256-cbc for 3s on 256 size blocks: 5207421 aes-256-cbc’s in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 1341335 aes-256-cbc’s in 3.05s
Doing aes-256-cbc for 3s on 8192 size blocks: 168932 aes-256-cbc’s in 3.07s
Doing aes-256-cbc for 3s on 16384 size blocks: 82658 aes-256-cbc’s in 3.00s
OpenSSL 1.1.1h-freebsd 22 Sep 2020
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang
The ‘numbers’ are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes 16384 bytes
aes-256-cbc 245029.07k 427327.38k 444366.59k 449645.68k 450732.93k 451422.89k