Installing Arch Linux with ZFS on a USB stick
Published on 2019-03-28. Modified on 2022-10-23.
In this tutorial I am going to show you how you can make a working Arch Linux USB stick with ZFS. This can be useful if you're running a homelab NAS and you want to boot of the USB stick and only use your drives for ZFS. It can also be used as a diagnostic or administration tool for dealing with ZFS on Arch Linux or some other Linux distribution.
Let me start by saying, running ZFS on Arch Linux is generally not a good idea. Not because it cannot be done, I have multiple servers that runs Arch Linux with ZFS, but the Arch Linux developers treat ZFS as a "second-class citizen" and as such, you either have to use something like zfs-dkms from AUR, use a non-official pacman repository like archzfs, or run your own custom repository. In either case, you need to keep an eye on things if you don't want stuff to break when you update the kernel.
TIP! Besides from all the Technical reasons to choose FreeBSD over GNU/Linux, if you want proper support for ZFS, run FreeBSD! FreeBSD treats ZFS as a true first-class citizen and no Linux distribution can compare in that regard. FreeBSD has tightly integrated ZFS into several of the system base tools. Even the top command displays ZFS ARC Stats.
Other and better options than Arch Linux are Void Linux, which has good support for ZFS. Also Debian and Devuan - if you enable their "contrib" repositories. Ubuntu also has good support for ZFS.
In this tutorial I am going to use two different methods in order to create a bootable USB stick. First I am going to use Archiso with the non-official archzfs respository to create an ISO which you can put unto an USB stick. Next I am going to use the method in which you mount an USB stick and install Arch Linux unto it from an already running Arch Linux machine with zfs-dkms
installed from AUR.
The Archiso method
You can keep an eye on the build process of archzfs on https://ci.archzfs.com/.
First install Archiso:
# pacman -S archiso
Then create a temporary working directory and enter into that:
$ mkdir tmpwork $ cd tmpwork
Copy the config files, syslinux, etc., into the directory (I am going to use the archlive version):
$ cp -r /usr/share/archiso/configs/releng/ archlive
Edit pacman.conf
and enter the archzfs-testing
repository from Archzfs in order to get the latest version of ZFS:
[archzfs-testing] Server = https://archzfs.com/$repo/$arch
The ordering within pacman.conf
matters. To give top priority to the Archzfs repository, place it above the other repository entries.
Edit packages.x86_64
to select which packages are to be installed on the live system image, listing packages line by line. I have added the following:
zfs-linux linux-headers neovim
Then become root, import the GPG key and build the ISO image:
$ su # curl -O https://archzfs.com/archzfs.gpg # pacman-key -a archzfs.gpg # pacman-key --lsign-key DDF7DB817396A49B2A2723F7403BD972F75D9D76 # mkarchiso -v ./
When Archiso is done, it will have created a directory called out
in which the resulting ISO file is placed. You can know use dd
to put it unto an USB stick.
# dd if=out/archlinux-2022.10.23-x86_64.iso of=/dev/sdX bs=4M oflag=direct status=progress
Make sure you change the sdX
to match your USB stick. You can check it by taking a look at the output of dmesg
as root, when you insert the stick to the computer.
The running Arch Linux installation method
I am going to use the ext4
filesystem for the USB stick. The Arch Linux wiki recommends disabling the journal when using ext4
on a USB stick, but since all decent flash devices perform internal wear leveling I think the benefit of running with the journal outweighs the risk of premature wearing out the USB stick.
I am going to use the MBR setup and not UEFI in this tutorial.
First we need to install the Arch Linux install scripts:
# pacman -S arch-install-scripts
Next, locate your USB device:
# fdisk -l
In my case it's the sdc
device.
Then partition the disk. If you're not comfortable using fdisk
then cfdisk
is a nice alternative partitioning tool.
# cfdisk /dev/sdc
Create a single bootable Linux partition, resulting in /dev/sdc1
in my case.
Create the ext4 filesystem:
# mkfs.ext4 /dev/sdc1
Use the following command instead if you want to disable the journal:
# mkfs.ext4 -O "^has_journal" /dev/sdc1
Then mount the USB stick:
# mount /dev/sdc1 /mnt
Then install the base package group, whatever custom packages you want, and set everything up:
# pacstrap /mnt base linux linux-firmware neovim cryptsetup grub # arch-chroot /mnt # ln -sf /usr/share/zoneinfo/Europe/Copenhagen /etc/localtime # hwclock --systohc # passwd root
Uncomment "en_US.UTF-8" and other needed locales in /etc/locale.gen
, and generate them with:
# locale-gen
Create the /etc/locale.conf
file and set the LANG variable accordingly:
LANG=en_US.UTF-8
Set the keyboard layout in /etc/vconsole.conf
KEYMAP=dk
Let's also get the base-devel
package:
# pacman -S --needed base-devel
I enable SSH in order to be able to log in to the computer booting of the USB stick. This is handy if the computer is a homelab NAS running of the USB stick (otherwise you don't need this):
# systemctl enable sshd
Install additional users and setup the network (you don't need to install additional users if you just want to use the USB stick as a diagnostic tool, however it makes using makepkg
a lot easier):
# useradd --create-home foo # passwd foo
Setup basic networking:
# vi /etc/systemd/network/wired.network [Match] Name=en* Name=eth* [Network] DHCP=yes
Make sure that the Name
options matches all the networking cards you use. By default udev assigns names to your network interface controllers using Predictable Network Interface Names, which prefixes interfaces names with en
(wired/Ethernet), wl
(wireless/WLAN), or ww
(WWAN). See systemd.net-naming-scheme(7).
Then enable the network:
# systemctl enable systemd-networkd.service # systemctl enable systemd-resolved
Then setup GRUB:
# grub-install --target=i386-pc /dev/sdc --recheck # grub-mkconfig -o /boot/grub/grub.cfg
I had some problems with GRUB not being able find the "root device" after using the USB stick on another computer than the one on which I installed the stick. This shouldn't be happening since the system is using UUID, which is a mechanism to give each filesystem a unique identifier. These identifiers are generated by filesystem utilities (e.g. mkfs.) when the device gets formatted and are designed so that collisions are unlikely. However, I found other users with similar problems: https://superuser.com/questions/769047/unable-to-find-root-device-on-a-fresh-archlinux-install
The solution is, as described in the link, to edit /etc/mkinitcpio.conf
and change the HOOKS line so that block
comes before autodetect
.
HOOKS="base udev block autodetect modconf filesystems keyboard fsck"
Then run mkinitcpio to regenerate the initramfs:
# mkinitcpio -p linux
Then it's time to install ZFS.
I am using the zfs-dkms
package from Arch Linux AUR and running makepkg
as the foo user I setup before:
# pacman -S dkms linux-headers git # su foo $ cd /home/foo $ git clone https://aur.archlinux.org/zfs-utils.git $ cd zfs-utils $ makepkg -s $ cd .. $ git clone https://aur.archlinux.org/zfs-dkms.git $ cd zfs-dkms $ makepkg -s $ cd .. $ exit # pacman -U zfs-utils/zfs-utils-x.x.x-x-x86_64.pkg.tar.xz # pacman -U zfs-dkms/zfs-dkms-x.x.x-x-x86_64.pkg.tar.xz
If you get the error:
==> Verifying source file signatures with gpg... zfs git repo ... FAILED (unknown public key DDF7DB817396A49B2A2723F7403BD972F75D9D76) ==> ERROR: One or more PGP signatures could not be verified!
Then manually import the key as explained in One or more PGP signatures could not be verified and Use a key server
$ gpg --recv-keys DDF7DB817396A49B2A2723F7403BD972F75D9D76
If that doesn't work, use:
# pacman-key -r DDF7DB817396A49B2A2723F7403BD972F75D9D76 # pacman-key --lsign-key DDF7DB817396A49B2A2723F7403BD972F75D9D76
You can add an IgnorePkg
entry to pacman.conf
to prevent these packages from upgrading when doing a regular update. Since the AUR is unsupported any packages you install are your responsibility to update, not pacman's. If packages in the official repositories are updated, you will need to rebuild any AUR packages that depend on those libraries.
# vi /etc/pacman.conf IgnorePkg=zfs-dkms zfs-utils
For ZFS to live by its "zero administration" namesake, the zfs daemon must be loaded at startup. A benefit to this is that it is not necessary to mount the zpool in /etc/fstab
The zfs daemon can import and mount zfs pools automatically. The daemon mounts the zfs pools reading the file /etc/zfs/zpool.cache
. For each pool you want automatically mounted by the zfs daemon execute:
# zpool set cachefile=/etc/zfs/zpool.cache <pool>
Enable the service so it is automatically started at boot time:
# systemctl enable zfs.target
Note: Beginning with ZOL version 0.6.5.8 the ZFS service unit files have been changed so that you need to explicitly enable any ZFS services you want to run. See https://github.com/archzfs/archzfs/issues/72 for more information.
In order to mount zfs pools automatically on boot (you only need that if you use the USB stick as a boot device for a NAS server) you need to enable the following services and targets:
# systemctl enable zfs-import-cache # systemctl enable zfs-mount # systemctl enable zfs-import.target
We're done. Exit the chroot and un-mount the USB stick:
# exit # umount /mnt
You should now have a working USB stick with Arch Linux and ZFS that you can use as a boot device or a diagnostic tool.