Installing UEFI ZFS Root on Ubuntu 20.04

There is a newer version of this guide for Ubuntu 20.04, but with native ZFS encryption.


Ubuntu 20.04 has a ZFS setup option as of 19.10. And frankly, you should use it instead of the manual installation procedure. However, manual installation does offer it's advantages - especially when it comes to pool layout and naming. If manual installation is needed, there is a great Root on ZFS installation guide that's part of ZFS-on-Linux project but its final ZFS layout is a bit too complicated for my taste. Here is my somewhat simplified version of the same, intended for a singe disk installations.

After booting into Ubuntu desktop installation we want to get a root prompt. All further commands are going to need root credentials anyhow.

Terminal
sudo -i

The very first step should be setting up a few variables - disk, pool, host name, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system.

Terminal
DISK=/dev/disk/by-id/ata_disk
POOL=Ubuntu
HOST=desktop
USER=user

General idea of my disk setup is to maximize amount of space available for pool with the minimum of supporting partitions. If you are planning to have multiple kernels, increasing boot partition size might be a good idea.

Terminal
blkdiscard $DISK

sgdisk --zap-all $DISK

sgdisk -n1:1M:+127M -t1:EF00 -c1:EFI $DISK
sgdisk -n2:0:+512M -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:0 -t3:8309 -c3:Ubuntu $DISK

sgdisk --print $DISK

Unless there is a major reason otherwise, I like to use disk encryption.

Terminal
cryptsetup luksFormat -q --cipher aes-xts-plain64 --key-size 512 \
--pbkdf pbkdf2 --hash sha256 $DISK-part3

I like to use disk name as the name of mapped (encrypted) luks device when I open it, but really anything goes.

Terminal
LUKSNAME=<code>basename $DISK</code>
cryptsetup luksOpen $DISK-part3 $LUKSNAME

Finally we're ready to create system ZFS pool.

Terminal
zpool create -o ashift=12 -o autotrim=on \
-O compression=lz4 -O normalization=formD \
-O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
-O canmount=off -O mountpoint=none -R /mnt/install $POOL /dev/mapper/$LUKSNAME
zfs create -o canmount=noauto -o mountpoint=/ $POOL/root
zfs mount $POOL/root

Assuming UEFI boot, two additional partitions are needed. One for EFI and one for booting. Unlike what you get with the official guide, here I don't have ZFS pool for boot partition but a plain old ext4. I find potential fixup works better that way and there is a better boot compatibility. If you are thinking about mirroring, making it bigger and ZFS might be a good idea. For a single disk, ext4 will do.

Terminal
yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot/

mkfs.msdos -F 32 -n EFI $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package.

Terminal
apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool is next. This will take a while.

Terminal
debootstrap focal /mnt/install/

zfs set devices=off $POOL

Our newly copied system is lacking a few files and we should make sure they exist before proceeding.

Terminal
echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials:

Terminal
mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we're ready to "chroot" into our new system.

Terminal
mount --rbind /dev /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys /mnt/install/sys
chroot /mnt/install \
/usr/bin/env DISK=$DISK POOL=$POOL USER=$USER LUKSNAME=$LUKSNAME \
bash --login

Let's not forget to setup locale and time zone.

Terminal
locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we're ready to onboard the latest Linux image.

Terminal
apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

Terminal
apt install --yes zfs-initramfs cryptsetup keyutils grub-efi-amd64-signed shim-signed tasksel

Since we're dealing with encrypted data, we should auto mount it via crypttab. If there are multiple encrypted drives or partitions, keyscript really comes in handy to open them all with the same password. As it doesn't have negative consequences, I just add it even for a single disk setup.

Terminal
echo "$LUKSNAME UUID=$(blkid -s UUID -o value $DISK-part3) none \
luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
cat /etc/crypttab

To mount boot and EFI partition, we need to do some fstab setup too:

Terminal
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
/boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
/boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
cat /etc/fstab

Now we get grub started and update our boot environment.

Terminal
KERNEL=<code>ls /usr/lib/modules/ | cut -d/ -f1 | sed &#039;s/linux-image-//&#039;</code>
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

Terminal
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
--recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it'll take a considerable amount of time.

Terminal
tasksel install ubuntu-desktop-minimal

Short package upgrade will not hurt.

Terminal
apt dist-upgrade --yes

We can omit creation of the swap dataset but I personally find a small one handy.

Terminal
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
-o sync=always -o primarycache=metadata -o secondarycache=none $POOL/swap
mkswap -f /dev/zvol/$POOL/swap
echo "/dev/zvol/$POOL/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too.

Terminal
rmdir /home
zfs create -o mountpoint=/home $POOL/home

The only remaining task before restart is to create the user, assign a few extra groups to it, and make sure its home has correct owner.

Terminal
sudo adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
passwd $USER

As install is ready, we can exit our chroot environment.

Terminal
exit

And cleanup our mount points.

Terminal
umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a

After the reboot you should be able to enjoy your installation.

Terminal
reboot


PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 21.10 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 22.10, 19.10, 19.04, and 18.10.

[2020-05-18: Changed boot partition size to 512M (was 384M). Reason is ever increasing size of kernel making it difficult to do future upgrades without going through cleanups if you use multiple kernels.]

[2020-06-27: Added blkdiscard and autotrim.]

7 thoughts to “Installing UEFI ZFS Root on Ubuntu 20.04”

  1. Thanks, you’ve got an awesome write-up!
    I easily replicated your approach in setup. One thing I’d like to improve is to use a native ZFS partition encryption instead of LUKS. Do you have an idea on how to make it work by any chance?

      1. I’d ideally like to do a full drive encryption and to decrypt the drive on boot with the key stored in TPM 2.0.
        The implementation would look like this: the boot partition is ext4 encrypted with LUKS and it is encrypted with the key from TPM. Then, the decrypted boot partition stores the key for the root ZFS volume, and the root volume is decrypted.

Leave a Reply to Alex Cancel reply

Your email address will not be published. Required fields are marked *