Installing UEFI ZFS Root on Ubuntu 19.04

As rumors of Ubuntu 19.04 including ZFS installer proved to be a bit premature, I guess it’s time for a slight adjustment to my previous ZFS instructions.

Again, all this is just a derivation on ZFS-on-Linux project’s instruction for older version.

As before, we first need to get into root prompt:

Terminal
sudo -i

Followed by getting a few basic packages ready:

Terminal
apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

Disk setup is quite simple with only two partitions:

Terminal
sgdisk --zap-all /dev/disk/by-id/ata_disk

sgdisk -n3:1M:+383M -t3:8300 -c3:Boot /dev/disk/by-id/ata_disk
sgdisk -n2:0:+128M -t2:EF00 -c2:EFI /dev/disk/by-id/ata_disk
sgdisk -n1:0:0 -t1:8300 -c1:Data /dev/disk/by-id/ata_disk

sgdisk --print /dev/disk/by-id/ata_disk

I believe full disk encryption should be a no-brainer so of course we set up LUKS:

Terminal
cryptsetup luksFormat -q --cipher aes-xts-plain64 --key-size 512 \
--pbkdf pbkdf2 --hash sha256 /dev/disk/by-id/ata_disk-part1
cryptsetup luksOpen /dev/disk/by-id/ata_disk-part1 system

Creating ZFS stays the same as before:

Terminal
zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 \
-O normalization=formD -O xattr=sa -O mountpoint=none system /dev/mapper/system
zfs create -o canmount=noauto -o mountpoint=/mnt/system/ system/root
zfs mount system/root

Getting basic installation on our disks follows next:

Terminal
debootstrap disco /mnt/system/
zfs set devices=off system
zfs list

And then we setup EFI boot partition:

Terminal
yes | mkfs.ext4 /dev/disk/by-id/ata_disk-part3
mount /dev/disk/by-id/ata_disk-part3 /mnt/system/boot/

mkdir /mnt/system/boot/efi
mkfs.msdos -F 32 -n EFI /dev/disk/by-id/ata_disk-part2
mount /dev/disk/by-id/ata_disk-part2 /mnt/system/boot/efi

We need to ensure boot partition auto-mounts:

Terminal
echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/ata_disk-part3) \
/boot ext4 noatime,nofail,x-systemd.device-timeout=1 0 1 >> /mnt/system/etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/ata_disk-part2) \
/boot/efi vfat noatime,nofail,x-systemd.device-timeout=1 0 1 >> /mnt/system/etc/fstab
cat /mnt/system/etc/fstab

Before we start using anything, we should prepare a few necessary files:

Terminal
echo "hostname" > /mnt/system/etc/hostname
sed 's/ubuntu/hostname/' /etc/hosts > /mnt/system/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/system/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/system/etc/netplan/

If you are installing via WiFi, you might as well copy your credentials:

Terminal
mkdir -p /mnt/system/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/system/etc/NetworkManager/system-connections/

With chroot we can get the first taste of our new system:

Terminal
mount --rbind --make-rslave /dev /mnt/system/dev
mount --rbind --make-rslave /proc /mnt/system/proc
mount --rbind --make-rslave /sys /mnt/system/sys
chroot /mnt/system/ /bin/bash --login

Now we can update our software:

Terminal
apt update

Immediately followed with locale and time zone setup:

Terminal
locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we install Linux image and basic ZFS boot packages:

Terminal
apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs

Since we’re dealing with encrypted data, our cryptsetup should be also auto mounted:

Terminal
apt install --yes cryptsetup keyutils

echo "system UUID=$(blkid -s UUID -o value /dev/disk/by-id/ata_disk-part1) \
none luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
cat /etc/crypttab

Now we get grub started:

Terminal
apt install --yes grub-efi-amd64

And update our boot environment again (seeing errors is nothing unusual):

Terminal
update-initramfs -u -k all

And then we finalize our grup setup:

Terminal
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
--bootloader-id=Ubuntu --recheck --no-floppy

Finally we get the rest of desktop system:

Terminal
apt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes

We can omit creation of the swap dataset but I always find it handy:

Terminal
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
-o sync=always -o primarycache=metadata -o secondarycache=none system/swap
mkswap -f /dev/zvol/system/swap
echo "/dev/zvol/system/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too:

Terminal
rmdir /home
zfs create -o mountpoint=/home system/home

Only remaining thing before restart is to create user:

Terminal
adduser user
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo user
chown -R user:user /home/user

As install is ready, we can exit our chroot environment and reboot unmount our new environment. If unmount fails, just repeat it until it doesn’t. :)

Terminal
exit
umount -R /mnt/system

Finally we can correct root’s mount point and reboot:

Terminal
zfs set mountpoint=/ system/root
reboot

Assuming nothing went wrong, your UEFI system is now ready.


[2019-10-27: Added –make-rslave]

14 thoughts to “Installing UEFI ZFS Root on Ubuntu 19.04”

  1. What about doing a:
    # zpool export system
    # zpool import -N system
    # zfs set mountpoint=/ system/root
    before the reboot?

        1. it’s busy because you need to umount /mnt/system/boot, proc, dev,sys etc.

          if anything of these fails to umount, try umount -l

  2. After mounting dev, proc and sys they should be made rslaves:

    mount –make-rslave /mnt/system/proc
    mount –make-rslave /mnt/system/dev
    mount –make-rslave /mnt/system/sys

    This makes unmounting them less of a fresh hell. Also I’m not sure if there’s a reason for this, but you have the partition numbers backwards. Shouldn’t the ESP partition be 1 and the ZFS partition be 2?

    1. ESP partition has to be physically at the beginning of disk. However, it can be any number between 1 to 128. It’s matter of personal preference which number you give it. I like to give number 1 to my biggest partition and work from there. However, all this would work with EFI as the first partition too, i.e.:

      sgdisk -n1:1M:+511M -t1:EF00 /dev/disk/by-id/ata_disk
      sgdisk -n2:0:0      -t2:8300 /dev/disk/by-id/ata_disk
      

      As long as -part1 and -part2 suffixes are swapped too, there is no difference.

  3. This was awesome, thanks! I did this before with 16.04 and 18.04, and probably could have used the 18.04 directions for 19.04, but it was nie to have 19.04 specific directions.

    Worked like a charm!

    -Mark

  4. Awesome for you to post this How-To for 19.04! This is my first attempt at ZFS on Root. I’ve followed the https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS and tried https://postgres.men/os/linux/ubuntu-18-04-root-on-zfs/ but without success. In my case I have a Dell T3500 with legacy bios (non UEFI). It’s ripe for use as a VM server for Docker, etc. Everything works well up till I install grub, it fails to install using the “BIOS” option given in the tuts. Would you have any suggestions or adjustment to your install how-to?

    Tony~

    1. Which version of Ubuntu you’re installing. If it’s 19.10, that could be the reason why it’s not working as ISO doesn’t contain all files needed to correctly set it up. I found that installing 19.04 and then upgrading to 19.10 works without issue. I also have a bit older BIOS boot guide at https://www.medo64.com/2019/01/setting-up-encrypted-ubuntu-18-10-zfs-desktop/. Take partition setup from it and follow newer guides when it comes to everything else.

      1. Thank you for your response!

        I’m installing on 19.04. Since this will be a VM server for Docker services I will avoid a short term release. Prior to seeing your response, I’ve successfully gotten a bootable system. What I found is that I needed to add “GRUB_DISABLE_OS_PROBER=true” to the /dev/default/grub file. The “apt install -y grub-pc” was, of course, using the default /etc/default/grub file. After the change I then reissued the the following commands where I was asked if I wanted to keep the original file. I am looking at testing “apt install -y grub-pc -o Dpkg::Options::=–force-confdef” to see if I can automate the selection for a scripted installation:

        apt reinstall -y grub-pc
        update-initramfs -u -k all
        update-grub
        grub-install $DISK1 (where $DISK1 = /dev/disks/by-id/)

        I’m now looking into why I am seeing the error “lz4 compression not supported” on boot. The system does boot and I am able to finalize the installation procedure. But errors are not something to ignore.

  5. Trying installing Ubuntu 19.10 with LUKS+ZFS but it fails at the `update-grub` step with this error:

    /usr/sbin/grub-probe: error: failed to get canonical path of `system/root`.

    Any suggestions?

  6. There seems to be issue with chroot unmounting cdrom too early and thus not all paths needed for manual installation are at their places. I am trying to figure workaround but nothing seems to work in UEFI mode. I ended up installing 19.04 and then upgrading to 19.10. :(

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.