Xen on ARM and Yocto

Follow these instructions to cross-compile a hypervisor and minimal Dom0 filesystem image containing the Xen tools for the ARM64 QEMU emulator. The instructions are similar for other ARM64 platforms.

Obtaining the source code

We will use the Yocto stable release, "Dunfell":

$ git clone -b dunfell http://git.yoctoproject.org/git/poky
$ cd poky
$ git clone -b dunfell http://git.openembedded.org/meta-openembeddded
$ git clone -b dunfell http://git.yoctoproject.org/git/meta-virtualization

Preparing the build tree

Initialize your shell to be ready to build - this will generate basic configuration files when you do this the first time:

 $ source ./oe-init-build-env

Edit conf/bblayers.conf, to add the source code layers needed. /scratch/repos/poky is the directory where you cloned the poky source code to - you will need to adjust the example paths here to match your directory layout:

BBLAYERS ?= " \
/scratch/repos/poky/meta \
/scratch/repos/poky/meta-poky \
/scratch/repos/poky/meta-yocto-bsp \
/scratch/repos/poky/meta-openembedded/meta-oe \
/scratch/repos/poky/meta-openembedded/meta-filesystems \
/scratch/repos/poky/meta-openembedded/meta-python \
/scratch/repos/poky/meta-openembedded/meta-networking \
/scratch/repos/poky/meta-virtualization \
 "

The conf/local.conf file contains instructions for the variables that it sets. You should review and make sure to set:

DL_DIR           -- set this to a local download directory for retrieved tarballs or other source code files
SSTATE_DIR       -- set to a local directory for build cache files to speed up subsequent builds
PACKAGE_CLASSES  -- package_ipk can be a good choice for package format

Then add the following to the same file, or make sure that the values here match if the variables are already present in the file:

MACHINE = "qemuarm64"
DISTRO = "poky"
IMAGE_FSTYPES += "cpio.gz"
QEMU_TARGETS = "i386 aarch64"
DISTRO_FEATURES += " virtualization xen"
BUILD_REPRODUCIBLE_BINARIES = "1"

This configuration will enable OpenEmbedded's support for reproducible builds. It also reduces the number of emulation platforms for QEMU to significantly reduce build time.

If you would like to build QEMU to provide PV backends, such as disk and 9pfs, then you need to add:

PACKAGECONFIG_pn-qemu += " virtfs xen fdt"

Sdl is enabled by default in the Xen build of QEMU but it is not actually necessary and can be disabled with:

PACKAGECONFIG_remove_pn-qemu += " sdl"

Building

$ bitbake xen-image-minimal

When the build is completed, the output image file will be in build/tmp/deploy/images.

Targetting hardware platforms

If you are targetting a specific ARM hardware platform, you will likely need to add the "BSP layer", which contains the MACHINE definition for the hardware, and any closely related software such as bootloader configuration. The MACHINE variable needs to be set to match the hardware definition. You may also add additional layers containing related support software, if any.

TODO: instructions in this section are still pending validation for Dunfell

eg. For the Raspberry Pi 4:

In your poky directory, add the BSP layer:

$ git clone -b dunfell https://git.yoctoproject.org/git/meta-raspberrypi

In local.conf, set:

MACHINE = "raspberrypi4-64"

In bblayers.conf, add:

BBLAYERS_append = " /scratch/repos/poky/meta-raspberrypi"

and then perform your build as before.

Building with a local copy of the Xen source code

TODO: instructions in this section are still pending validation for Dunfell

If you are building a local copy of a Xen source tree, you can add to local.conf:

INHERIT_pn-xen += "externalsrc"
INHERIT_pn-xen-tools += "externalsrc"
EXTERNALSRC_pn-xen = "/scratch/repos/xen"
EXTERNALSRC_pn-xen-tools = "/scratch/repos/xen"
EXTERNALSRC_BUILD_pn-xen = "/scratch/repos/xen"
EXTERNALSRC_BUILD_pn-xen-tools = "/scratch/repos/xen"