Medo's Home Page

Installing UEFI ZFS Root on Ubuntu 21.10

2021-10-242021-10-22 Josip Medved Leave a comment

Before reading further you should know that Ubuntu has a ZFS setup option since 19.10. You should use it instead of the manual installation procedure unless you need something special. In my case that special something is the native ZFS encryption and custom partitioning I find more suitable for laptop.

After booting into Ubuntu desktop installation we want to get a root prompt. All further commands are going to need root credentials anyhow.

Terminal 
sudo -i

The very first step should be setting up a few variables – disk, pool, host name, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system. I like to use upper-case for ZFS pool as that’s what will appear as password prompt. It just looks nicer and ZFS doesn’t care either way.

Terminal 
DISK=/dev/disk/by-id/ata_disk
POOL=Ubuntu
HOST=desktop
USER=user

General idea of my disk setup is to maximize amount of space available for pool with the minimum of supporting partitions. If you are not planning to have multiple kernels, decreasing boot partition size might be a good idea (512 MB is ok).

Terminal 
blkdiscard -f $DISK

sgdisk --zap-all                     $DISK

sgdisk -n1:1M:+63M -t1:EF00 -c1:EFI  $DISK
sgdisk -n2:0:+960M -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:0     -t3:BF00 -c3:Root $DISK

sgdisk --print                       $DISK

Finally we’re ready to create system ZFS pool. Note that you need to encrypt it at the moment it’s created.

Terminal 
zpool create -o ashift=12 -o autotrim=on \
    -O compression=lz4 -O normalization=formD \
    -O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
    -O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
    -O canmount=off -O mountpoint=none $POOL $DISK-part3

On top of this encrypted pool we could create our root set (as I did in previous guides) or just use default dataset itself. I found that actually works better for me. In either case, our new root file system will end up at /mnt/install.

Terminal 
zfs set canmount=noauto $POOL
zfs set mountpoint=/ $POOL

mkdir /mnt/install
mount -t zfs -o zfsutil $POOL /mnt/install

Assuming UEFI boot, two additional partitions are needed. One for EFI and one for booting. Unlike what you get with the official guide, here I don’t have ZFS pool for boot partition but a plain old ext4. I find potential fixup works better that way and there is a better boot compatibility. If you are thinking about mirroring, making it bigger and ZFS might be a good idea. For a single disk, ext4 will do.

Terminal 
yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot/

mkfs.msdos -F 32 -n EFI $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package.

Terminal 
apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool is next. This will take a while.

Terminal 
debootstrap $(basename `ls -d /cdrom/dists/*/ | grep -v stable | head -1`) /mnt/install/

We can use our live system to update a few files on our new installation.

Terminal 
echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials. Don’t worry if this returns errors – that just means you are not using wireless.

Terminal 
mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we’re ready to “chroot” into our new system.

Terminal 
mount --rbind /dev  /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys  /mnt/install/sys
chroot /mnt/install \
    /usr/bin/env DISK=$DISK POOL=$POOL USER=$USER \
    bash --login

Let’s not forget to setup locale and time zone.

Terminal 
locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we’re ready to onboard the latest Linux image.

Terminal 
apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

Terminal 
apt install --yes zfs-initramfs grub-efi-amd64-signed shim-signed tasksel

To mount boot and EFI partition, we need to do some fstab setup.

Terminal 
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
cat /etc/fstab

Now we get grub started and update our boot environment.

Terminal 
KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

Terminal 
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
    --recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it’ll take a considerable amount of time.

Terminal 
tasksel install ubuntu-desktop-minimal

Short package upgrade will not hurt.

Terminal 
apt dist-upgrade --yes

We can omit creation of the swap dataset but I personally find a small one handy.

Terminal 
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
    -o sync=always -o primarycache=metadata -o secondarycache=none $POOL/Swap
mkswap -f /dev/zvol/$POOL/Swap
echo "/dev/zvol/$POOL/Swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too but I usually skip it these days. I just proceed to create the user, assign a few extra groups to it, and make sure its home has correct owner.

Terminal 
adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
passwd $USER

As install is ready, we can exit our chroot environment.

Terminal 
exit

And cleanup our mount points.

Terminal 
umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
umount /mnt/install
zpool export -a

After the reboot you should be able to enjoy your installation.

Terminal 
reboot

PS: There are versions of this guide for other Ubuntu versions: 20.04 (native), 20.04 (luks), 19.10, 19.04, and 18.10. They are all slightly different since my goals changed over time.

ZFS Pool on SSD

2021-10-162021-10-17 Josip Medved Leave a comment

I am a creature of habit. Long time ago I found ZFS setup that works for me and didn’t change much since. But sometime I wonder if those settings still hold with SSDs in game. Most notably, are 4K blocks still the best?

Since I already “had” to update my desktop to Ubuntu 21.10, I used that opportunity to clear my disks and have it installed from scratch. And it would be a shame not to run some tests first on my XPG SX6000 Pro – SSD I use for pure data storage. After trimming this DRAM-less SSD, I tested the pool across multiple recordsize values and at ashift values of 12 (4K block) and 13 (8K block).

My goal was finding a good default settings for both bulk storage and virtual machines. Unfortunately, those are quite opposite requirements. Bulk storage benefits greatly from good sequential access while virtual machines love random IO more. Fortunately, with ZFS, one can accomplish both using two datasets with different recordsize values. But ashift value has to be the same.

Due to erase block sizes getting larger and larger, I expected performance to be better with 8K “sectors” (ashift=13) than what I usually used (ashift=12). But I was surprised.

First of all, results were all over the place but it seems that ashift=12 is still a valid starting point. It might be due to my SSD having smaller than expected erase page but I doubt it. My thoughts go more toward SSDs being optimized for the 4K load. And the specific SSD I used to test with is DRAM-less thus allowing any such optimizations to be even more visible.

Optimizations are probably also the reason for 128K performing so well in the random IO scenarios. Yes, for sequential access you would expect it, but for random access it makes no sense how fast it is. No matter what’s happening, it’s definitely making recordsize=128K still the best general choice. Regardless, for VMs, I created a sub-dataset with much smaller 4K records (and compression off) just to lower write-amplification a bit.

The full test results are in Google Sheets. For testing I used fio’s fio-rand-RW.fio and fio-seq-RW.fio profiles.

Overescaping By Default

2021-10-102021-10-09 Josip Medved Leave a comment

Writing JSON has became trivial in C# and there’s no class I like better for that purpose than Utf8JsonWriter. Just look at a simple example:

Code 
var jsonUtf8 = new Utf8JsonWriter(Console.OpenStandardOutput(),
                                  new JsonWriterOptions() { Indented = true });
jsonUtf8.WriteStartObject();
jsonUtf8.WriteString("Test", "2+2");
jsonUtf8.WriteEndObject();
jsonUtf8.Flush();

This simple code will produce perfectly valid JSON:

JSON 
{
  "Test": "2\u002B2"
}

While valid, you’ll notice this is slightly different than any other programming language would do. A single plus character became escape sequence \u002B.

In their eternal wisdom, .NET architects decided that, by default, JSON should be over-escaped and they “explained” their reasoning in the ticket. Essentially they did it out of abundance of caution to avoid any issues if someone puts JSON where it might not be expected.

Mind you, in 99% of cases JSON is used in HTTP body and thus doesn’t need this but I guess one odd case justifies this non-standard but valid output in their minds. And no, other JSON encoders don’t behave this way either. Only .NET as far as I can tell.

Fortunately, some time later, they also implemented what I (alongside probably 90% of developers) consider the proper JSON encoder which escapes just mandatory characters and leaves the rest of text alone. It just requires a small extra parameter.

Code 
var jsonUtf8 = new Utf8JsonWriter(Console.OpenStandardOutput(),
                                  new JsonWriterOptions() { Indented = true,
                                    Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping});
jsonUtf8.WriteStartObject();
jsonUtf8.WriteString("Test", "2+2");
jsonUtf8.WriteEndObject();
jsonUtf8.Flush();

Using UnsafeRelaxedJsonEscaping is not unsafe despite it’s name; darn it, it’s not even relaxed as compared to the specification. It’s just a properly implemented JSON encoder without any extra nonsense thrown in.

Framework Expansion Board

2021-10-032021-10-04 Josip Medved Leave a comment

One of most exciting recent developments in laptop world for me is definitely the framework laptop. A major component of that concept are its expansion cards. And, of course, you can build your own.

This repository is quite encompassing if you’re using KiCAD. However, for those who love nicer tools (ehm, DipTrace), it’s annoying to find that there is no board size specification in human readable format (and no, KiCAD XML is not). So I decided to figure it out.

To cut the long story short, here are the board outline points for the expansion card PCB:

(0.0, 0.0)
(26.0, 0.0)
(26.0, 26.5)
(25.0, 26.5)
(25.0, 30.0)
(17.7, 30.0)
(17.7, 28.0)
(16.0, 28.0)
(16.0, 29.0)
(10.0, 29.0)
(10.0, 28.0)
(8.3, 28.0)
(8.3, 30.0)
(1.0, 30.0)
(1.0, 26.5)
(0.0, 26.5)

In order to make it slightly nicer to handle, each corner is additionally rounded with a 0.3 mm radius.

And let’s not forget two holes at (1.7, 10.5) and (24.3, 10.5), both with a 2.2 mm diameter and 4.9 mm keepout region.

With that information in hand, one can create PCB board in any program they might prefer. Of course, I already did so for DipTrace and you download the files here.

And yes, PCB is just a first step in a development process. What I found the hardest is actually getting appropriate connectors for the enclosure as there’s not too much height to work with.

PS: No, I do not own framework laptop at this time. I am waiting for 15.6″ model as 13.5″ is simply too small for me when not used external monitor.