Boot many distributions from USB


I often do hardware tests and compare Linux desktops. So I need many distributions bootable but also isolated. A handful of pen drives is the easiest solution. The following sections describe how to go one step further and bring them to a single USB disk.

What is needed to accomplish the tasks:

I spent a few hours looking for the right enclosure. In general it can't be booted connected to USB 3.0. Nevertheless I don't want to spend hours waiting for the installation. Maybe I also want to put the disk into the mounting bay of one of my desktops. I settled with a rather common Icy Box enclosure and an average Wester Digital 1TByte 2.5" drive. While the enclosure is capable of USB attached SCSI or short UAS (VL711 chipset) the harddisk was at hand and doesn't suffer from power down issues like Seagate disks (built into Intenso enclosures).

Now order is important. Installers of Linux also try to be clever and run various flavors of so called os-prober[1]. This piece of strangeness scans harddisks for other operating systems. In combination with Grub – the bootloader to be installed – this creates only a mess of partly bootable installations. Thus the following order:

  1. Partition the disk, USB bootable
  2. Install strange flavors of Linux like Ubuntu or Mint
  3. Install serious flavors of Linux like Debian
  4. Install reliable true Arch Linux
  5. Repair /boot/grub/grub.cfg by hand

Partitioning the disk is an odd thing and very likely to get wrong. There is no help out there except a few technical constraints and personal experience. With Grub the disk requires a very tiny partition at the start to store Grub itself. Do not mix this up with Linux mountpoint /boot. This is the second partition to be created as a VFAT (FAT32) partition. It wouldn't be necessary as VFAT if you don't run Windows from the same drive. I simply can install Linux with a VFAT boot. With Grub in charge of getting things started one can also use ext2 (or ext4 – a bit overdone for 512MByte). Finally there'll be many Partitions, one for each distribution. I also omit any Swap-partition. Linux runs fine without it and it is about a USB bootable toy not a desktop that needs hibernation or solve tasks that require swap. Use your favorite tool. I prefer fdisk but gdisk, gparted, parted or any other partitioning tool does well.

Check your target harddisk's device name and model twice. If you plug in the USB disk use lsblk and dmesg to verify, what device name your USB harddisk has. In the following I use /dev/sdX as a defunct placeholder. For you it could be /dev/sdc or any other device. Also write down the device's manufacturer string, e.g. ST500LM000-1EJ162 (smartctl -a /dev/sdX | grep 'Device Model') The various installers will change the device names and you don't want to break the installation of your primary host by changing its partion table(s).

Ubuntu is to be installed first. It takes about 2hours to accomplish this task. Get the ISO first, write it to the thumb drive with dd and boot it – the installation target disk plugged into a USB 3 socket. I used 20.04-something and checked third party/ multi media packages. In turn it crashed with a known issue at the end. The installation itself was absolutely usable. Installing Ubuntu is one of the easier tasks. It definitely leaves a broken Grub and includes os-prober actions. It is also incapable of using partition #2 as mount point /boot.

After shutdown Linux Mint is next. It is the least interactive installation and there is nothing to select or choose – except which partition is to be used. Like Ubuntu Grub is still broken and partition #2 cannot be used as mountpoint /boot. Only partition #4 is used as installation target and like before the installer warns about unchanged partition table and file systems.

Again after shutdown Debian is the third distribution. I prefer the text installer which works the same as the graphical ones of Ubuntu and Mint. Do not forget to select a network mirror. This can be done afterwards, too but requires editing from hand. Without a network mirror all invocations of apt or apt-get end with the strange error message about broken held packages.

Shutdown a last time and start Arch Linux installation to fix all Grub issues. It is important to start pacstrap with additional packages dhcpcd, vim (and/ or) nano and openssh. Grub will be added later in the chroot environment. All previous distributions came with an installer which is quite magic and often hard to understand. They have documentation full of technical terms and without explanations why this parameter is necessary or why another partioning schema is sub-optimal. Arch instead boots itself and gives you all the commands necessary to simply copy or bootstrap things. After Gentoo this is my next choice of Linux distributions and much easier to handle than the graphic ones – either it works or gives a searchable error message that takes you to their wiki. The only downside is that kernel and initramfs come without a version number.

After following all the initial steps from the Wiki except partitioning also create a directory /mnt/boot and mount partition #2 to this directory. Do this before arch-chroot. Now follow all the remaining steps and configure host name and such. Finally get Grub and os-prober with pacman -S grub os-prober and install Grub to the USB disk and run grub-mkconfig -o /boot/grub/grub.cfg. It will detect all the other operation systems already installed and create menu entries accordingly. I personally went even further and moved all files with version numbers to Arch's /boot like so (sample for Ubuntu on 3rd partition with kernel version 5.4.0-42, repeat for Mint and Debian):

archroot> mount /dev/sdX3 /mnt
archroot> mv /boot/*-5.4.* /boot
archroot> umount /mnt

After this I ran grub-mkconfig like stated above and correct all menu entries afterwards. List all UUID with blkid use grep to filter out the UUIDs only and/ or append them to /boot/grub/grub.cfg: blkid >> /boot/grub/grub.cfg. With your favorite command line text editor open /boot/grub/grub.cfg and look for menuentry and inside its curly braces for root=UUID=. There'll be one per each distribution and a few extra. Grub assumed all of them reside in Arch's partition #6. Instead replace each entry's UUID= with the correct one from the end of the file. This way Arch's Grub installation reads kernel and initramfs according to each menuentry and hands over control with the apropriate root partition's UUID – that easy. Here's a typical menuentry UUID marked with v/^.

menuentry 'Gentoo GNU/Linux, with Linux 5.4.48-gentoo-sec' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.4.48-gentoo-sec-advanced-b00e271d-8c78-4175-a337-358d6f3d967d' {
		insmod gzio
		insmod part_gpt
		insmod fat
		set root='hd0,gpt1'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  4E3D-1ADC
		  search --no-floppy --fs-uuid --set=root 4E3D-1ADC
		echo	'Loading Linux 5.4.48-gentoo-sec ...'
		#                                            vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
		linux	/vmlinuz-5.4.48-gentoo-sec root=UUID=b00e271d-8c78-4175-a337-358d6f3d967d ro  resume=/dev/sda2
		#                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
		echo	'Loading initial ramdisk ...'
		initrd	/amd-uc.img /initramfs-5.4.48-gentoo-sec.img


Fix grub.cfg, move other distribution's kernel to /boot-mountpoint. Also possible to chain Grub and run independent/ isolated distributions. Still Arch Linux the primary. All installers want to install boot loader thus each break it.

Why wasn't a virtual or containerized installation enough? I needed hardware access and a fully blown linux environment. This includes especially desktops, using a touch monitor, the I²C-bus or access hardware through smartctl or the direct network stack. There are also limitations. It is not possible to upgrade the distributions easily. Most of them will upgrade linux-firmware or the Kernel. After installation of these packages triggers do housekeeping work. This includes creating a new initramfs or invoking grub-mkconfig.

As a final note you also have an illustration of all security issues that arise from unencrypted operating system resources and user content. Only one of the distributions can be started at a time. Nevertheless you gain root access to all other distributions. Simply mount the partition with sudo, copy files or edit configuration. Except Arch all other distributions create a regular user with sudo permissions. Ubuntu makes it a bit harder to use ls on root-only directories. But at least user content is bound to a numeric user id (UID). All start with 1000 and same UID means same user. A home directory for UID 1000 under Debian has the same owner as the Linux Mint's UID 1000. Easy to prepare a little thumb drive with Arch Linux, fancy tools added like NTFS write support and network monitoring and off you go. No password to protect a host's boot configuration: plug it in and have fun. No encrypted disks: feel free to edit anything.

  1. os-prober – Debian Repository