Jan 17, 2024 by Thibault Debatty | 2761 views
https://cylab.be/blog/319/understand-and-implement-pxe-network-boot
PXE is a powerful system that allows to boot a system from the network. The applications are numerous : install a system from the network, create diskless or kiosk systems. But the system is pretty complex! In this blog post we will explain how network booting works, and show an example to allow interactive installation of Ubuntu 22.04 server from the network...
This is how PXE network booting usually works:
In the rest of this blog post:
.iso
file, which will be downloaded using https
), so we can install Ubuntu 22.04 server directly from the network.First because there are 3 stages involved:
Second, because a lot of variations are possible:
Boot Server Host Name
, and 67, Bootfile Name
, but there are other possibilities, like DHCP option 43 Vendor Specific Configuration
. When used with PXE, this option allows to list multiple boot servers and define different menu entries./home
from the networkThird, because of the differences between BIOS and UEFI.
UEFI is an improvement of the BIOS [6]. Unlike BIOS, UEFI supports boot media larger than 2TB thanks to the GPT partitioning scheme. Also, UEFI is more flexible than BIOS regarding boot options: bootstrap code is no longer required at fixed locations (the MBR of the partition), but can be any file. However, the bootstrap file must be formatted according to the Portable Executable / Common Object File Format (PE/COFF), which means the DHCP server must lead the PXE to a different intermediate bootloader depending on the client boot mode (BIOS or UEFI).
In the example below, we will assume that the client computers are configured in BIOS boot mode. We will cover UEFI network booting in another blog post...
Finally, because of incorrect implementations of PXE. Some network devices have buggy PXE implementations, that do not entirely respect the PXE standard. Some of these issues can be circumvented by adding appropriate DHCP configuration [7, 8].
Because PXE is pretty limited and only supports TFTP, and TFTP in turn is also pretty limited:
So although TFTP (and PXE) could theoretically be used to directly load a full-blown OS, it is not feasible in real-life, which means they are only used to load a more evolved and robust bootloader...
There is a project called iPXE [2] that aims to implement an improvement version of PXE directly on the network card. However, at the time of writing iPXE is not integrated in existing network cards, which means you have to flash the PXE ROM on your network card yourself. Once done, you typically don't need an intermediate bootloader anymore.
You can also chainload into iPXE to obtain the features of iPXE without the hassle of reflashing, but this means you are simply using iPXE as intermediate bootloader...
So, the first step is to configure options 66 and 67 on your DHCP server. We show this below for OPNsense and for ISC DHCP server (the default DHCP server on a Ubuntu server)
On an OPNsense router, head to Services > DHCPv4 > LAN > TFTP Server
and fill the fields TFTP Hostname
and Bootfile
. In this example, 192.168.1.10
is the address of the TFTP server:
If not done yet, install the DHCP server:
sudo apt install isc-dhcp-server
Then modify the configuration file /etc/dhcp/dhcpd/conf and configure your subnet according to your needs:
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.100 192.168.1.199
option routers 192.168.1.1;
option domain-name-server 9.9.9.9;
# option 66
option tftp-server-name "192.168.1.10";
# option 67
option bootfile-name "pxelinux.0";
}
In this example, 192.168.1.10
is the address of the TFTP server. Don't forget to restart the server:
sudo service isc-dhcp-server restart
We can now install the tftp server:
sudo apt install tftpd-hpa
You should also increase the verbosity by modifying the config file /etc/default/tftpd-hpa and and the option --verbose
:
TFTP_OPTIONS="--secure --verbose"
And don't forget to restart the server:
sudo service tftpd-hpa restart
As you could also see from the file, the root directory for serving files is /srv/tftp
.
PXELINUX [1] is a Syslinux derivative for booting from a network server using a network ROM conforming to the Intel PXE (Pre-Execution Environment) specification.
We will need files from Syslinux and PXELINUX, so we install both:
sudo apt install syslinux pxelinux
If you remember from the TFTP section, the root directory is /srv/tftp
. And if you remember from the DHCP section, PXE will try to download and execute pxelinux.0
, so we have to copy this file in that folder. We will also present a menu to the user, which means pxelinux itself will try to download additional code files, which we must also copy in the tftp
directory:
sudo cp /usr/lib/PXELINUX/pxelinux.0 /srv/tftp
sudo cp /usr/lib/syslinux/modules/bios/ldlinux.c32 /srv/tftp
# to display the menu:
sudo cp /usr/lib/syslinux/modules/bios/menu.c32 /srv/tftp
sudo cp /usr/lib/syslinux/modules/bios/libutil.c32 /srv/tftp
The default location for the menu file that PXELINUX will present to the user is pxelinux.cfg/default
(on the same TFTP server) [3], so we can create the file and directory:
sudo mkdir /srv/tftp/pxelinux.cfg
nano /srv/tftp/pxelinux.cfg/default
with the following content:
UI menu.c32
LABEL Install Ubuntu 22.04 server
KERNEL ubuntu2204/vmlinuz
INITRD ubuntu2204/initrd
APPEND root=/dev/ram0 ip=dhcp cloud-init=disabled url=https://releases.ubuntu.com/jammy/ubuntu-22.04.3-live-server-amd64.iso
LABEL Boot from disk
LOCALBOOT 0
Most lines are self-explanatory. The syntax and other possibilities can be found on the wiki of Syslinux [4].
As you can see from the kernel command line (APPEND
), I disabled cloud-init. You should of course tune these parameters according to your needs...
As we can read from the menu entry, PXELINUX will try to download vmlinuz
(the kernel) and initrd (the root filesystem) [5] from directory ubuntu2204 (on the same TFTP server). So we have to create the corresponding directory:
sudo mkdir /srv/tftp/ubuntu2204
And we will extract vmlinuz
and initrd
from the installation iso
:
curl -O https://releases.ubuntu.com/jammy/ubuntu-22.04.3-live-server-amd64.iso
sudo mount -o ro,loop ubuntu-22.04.3-live-server-amd64.iso /mnt
sudo cp /mnt/casper/vmlinuz /srv/tftp/ubuntu2204
sudo cp /mnt/casper/initrd /srv/tftp/ubuntu2204
We can now test our setup. On the server, /var/log/syslog
is a good source information, and help debut eventual problems:
tail -f /var/log/syslog
If another computer is connected on the network, and configured to boot from the network (check the BIOS) for this, the PXELINUX should be displayed, allowing to install Ubuntu...
In the screenshot above, we can see on the server (left):
We can also monitor the process by recording packets with something like tcpdump
or wireshark
. On the screenshot below we can see the options 66 and 67 transmitted by the DHCP server.
You can also download the complete pcap: pxe-boot.pcap
This blog post is licensed under CC BY-SA 4.0