Installing UEFI ZFS Root on Ubuntu 21.10

Before reading further you should know that Ubuntu has a ZFS setup option since 19.10. You should use it instead of the manual installation procedure unless you need something special. In my case that special something is the native ZFS encryption and custom partitioning I find more suitable for laptop.

After booting into Ubuntu desktop installation we want to get a root prompt. All further commands are going to need root credentials anyhow.

Terminal
sudo -i

The very first step should be setting up a few variables - disk, pool, host name, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system. I like to use upper-case for ZFS pool as that's what will appear as password prompt. It just looks nicer and ZFS doesn't care either way.

Terminal
DISK=/dev/disk/by-id/ata_disk
POOL=Ubuntu
HOST=desktop
USER=user

General idea of my disk setup is to maximize amount of space available for pool with the minimum of supporting partitions. If you are not planning to have multiple kernels, decreasing boot partition size might be a good idea (512 MB is ok).

Terminal
blkdiscard -f $DISK

sgdisk --zap-all $DISK

sgdisk -n1:1M:+63M -t1:EF00 -c1:EFI $DISK
sgdisk -n2:0:+960M -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:0 -t3:BF00 -c3:Root $DISK

sgdisk --print $DISK

Finally we're ready to create system ZFS pool. Note that you need to encrypt it at the moment it's created.

Terminal
zpool create -o ashift=12 -o autotrim=on \
-O compression=lz4 -O normalization=formD \
-O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
-O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
-O canmount=off -O mountpoint=none $POOL $DISK-part3

On top of this encrypted pool we could create our root set (as I did in previous guides) or just use default dataset itself. I found that actually works better for me. In either case, our new root file system will end up at /mnt/install.

Terminal
zfs set canmount=noauto $POOL
zfs set mountpoint=/ $POOL

mkdir /mnt/install
mount -t zfs -o zfsutil $POOL /mnt/install

Assuming UEFI boot, two additional partitions are needed. One for EFI and one for booting. Unlike what you get with the official guide, here I don't have ZFS pool for boot partition but a plain old ext4. I find potential fixup works better that way and there is a better boot compatibility. If you are thinking about mirroring, making it bigger and ZFS might be a good idea. For a single disk, ext4 will do.

Terminal
yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot/

mkfs.msdos -F 32 -n EFI $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package.

Terminal
apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool is next. This will take a while.

Terminal
debootstrap $(basename `ls -d /cdrom/dists/*/ | grep -v stable | head -1`) /mnt/install/

We can use our live system to update a few files on our new installation.

Terminal
echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials. Don't worry if this returns errors - that just means you are not using wireless.

Terminal
mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we're ready to "chroot" into our new system.

Terminal
mount --rbind /dev /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys /mnt/install/sys
chroot /mnt/install \
/usr/bin/env DISK=$DISK POOL=$POOL USER=$USER \
bash --login

Let's not forget to setup locale and time zone.

Terminal
locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we're ready to onboard the latest Linux image.

Terminal
apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

Terminal
apt install --yes zfs-initramfs grub-efi-amd64-signed shim-signed tasksel

To mount boot and EFI partition, we need to do some fstab setup.

Terminal
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
/boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
/boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
cat /etc/fstab

Now we get grub started and update our boot environment.

Terminal
KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

Terminal
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
--recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it'll take a considerable amount of time.

Terminal
tasksel install ubuntu-desktop-minimal

Short package upgrade will not hurt.

Terminal
apt dist-upgrade --yes

We can omit creation of the swap dataset but I personally find a small one handy.

Terminal
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
-o sync=always -o primarycache=metadata -o secondarycache=none $POOL/Swap
mkswap -f /dev/zvol/$POOL/Swap
echo "/dev/zvol/$POOL/Swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too but I usually skip it these days. I just proceed to create the user, assign a few extra groups to it, and make sure its home has correct owner.

Terminal
adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
passwd $USER

As install is ready, we can exit our chroot environment.

Terminal
exit

And cleanup our mount points.

Terminal
umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
umount /mnt/install
zpool export -a

After the reboot you should be able to enjoy your installation.

Terminal
reboot

PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 22.04 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, 19.04, and 18.10.

3 thoughts to “Installing UEFI ZFS Root on Ubuntu 21.10”

  1. Hey, great guides.

    So far, has the manual install method broken anything as compared to a standard ubiquity bloatware install?

    Does zsys get installed? Happy to not have that kill the bpool on the standard install.

    Thanks

  2. I haven’t had any issues.

    Yes, there are many applications that are not installed by default but it’s easy enough to install them manually. And no, zsys is not installed by default. :)

  3. Hey Josip.

    I used your guide but on a qemu/kvm vm instead of baremetal. A few tips:

    #systemctl start spice-vdagent
    ..to enable cut and paste buffer between host and guest

    virtio doesn’t provide a nice /dev/disk/by-id path for virtual disks but you can use the EFI, Boot, and Root partition labels and /dev/disk/by-partlabel path, or even make the labels more descriptive e.g. ata-QEMU_VIRTIO_QM00001-part1, part2, part3 etc

    Everything installed and Ubuntu 21.10 booted but it seems that it doesn’t play nicely with spice and qxl graphics in qemu.

    First issue is that the root ZFS password input screen just comes up blank. You can enter a password (blind) and get to the desktop but you don’t see anything

    Second issue is that the desktop then crashes to a blank screen after it goes to screensave, with this kernel error in the syslog:

    [ 1611.159379] [TTM] Buffer eviction failed
    [ 1611.159461] qxl 0000:00:01.0: object_init failed for (9637888, 0x00000001)
    [ 1611.159471] [drm:qxl_alloc_bo_reserved [qxl]] *ERROR* failed to allocate VRAM BO

    so you really need to install ssh and grab the vm IP previously so you can shutdown gracefully

    I found a kinda workaround with spice, you need to shutdown then change the Video mode in qemu from QXL to Virtio and boot. This also fixes the password entry screen.

    Appreciate these issues are essentially VM related but these tips might help someone else doing a trial run in qemu.

Leave a Reply

Your email address will not be published. Required fields are marked *