Distros

LINUX DISTRIBUTIONS: WHAT TO TEST AFTER PACKAGING

mkinitrd, mkinitramfs and dracut

mkinitrd, mkinitramfs or dracut should be able to generate bootloader entries for Xen, see this page for an example.

Dom0 support in Linux

The Linux kernel packaged with the distribution should be compiled with Xen support, see this page. Booting the machine with Xen and the default kernel should complete successfully. If a user installs Xen dom0 packages, then the default boot entry should be changed to one that brings up the Xen kernel and dom0 successfully.

init scripts: xencommons

Xen relies on an init script called xencommons to initialize all the services. It should be executed by default at boot time.

qemu-xen-traditional and qemu-xen

The two device model binaries Xen relies on should be installed in the location where Xl expect them to be, that by default is /usr/lib/xen/bin, and can be changed modifying tools/libxl/_libxl_paths.h.

Xen udev scripts

The Xen udev scripts should be installed correctly, see /etc/udev/rules.d/xen-backend.rules: after manually creating the default bridge named xenbr0, starting a guest with a vif, the vif should be added to the bridge automatically.

External Links

patch to reduce spurious Xen entries in grub menu

LINUX DISTRIBUTIONS: SUGGESTED CONFIGURATIONS

LVM

Setting up lvm is strongly suggested: storing raw disk images in lvm volumes is the fastest possible configuration as you can exploit the in-kernel disk backend (CONFIG_XEN_BLKDEV_BACKEND in the Linux kernel config).

Then you can use:

  disk = [ 'phy:/dev/vol0/lvm_volume0,xvda,w' ]

in your VM config file.

Memory

If the dom0 Host-to-be has more memory than a typical laptop/desktop system has, then do not rely on dom0 ballooning. Instead set the dom0 ram to something between 1 and 4GB using dom0_mem=1024M

1GB is enough for a pretty large host, more will be needed if you expect your users to use advanced storage types as ZFS or distributed filesystems.

The reason to not give all ram to dom0 is that it takes time to re-purpose it went domUs are started. On a host with 128GB Ram you're getting things stuck for minutes while dom0 is ballooning down to make space.

In pratice, this is a pure desktop feature.

LINUX DISTRIBUTIONS: USER EXPERIENCE

Meta packages

For convenience, a Xen meta-package should be provided that contains the hypervisor, the xen tools and documentation.

Installation

The basic flow of Xen installation should be:

• Install distro;
• Install Xen packages or the Xen meta-package (as part of this the bootloader entries would be updated);
• Update-grub (or equivalent);
• Reboot, and you are ready to go.

Man pages

Xen comes with man pages, see

• Xen Man Pages (via Wiki)
• Xen Man pages for unstable

They should be installed automatically with the xen-tools package and made available to users. The source and make files of the man pages (and documentation in general) are in xen-unstable.hg/docs

LINUX DISTRIBUTIONS: THINGS TO TEST IN VIRTUAL MACHINES

boot

The distro should be able to update a kernel and build a valid boot config in a VM. Old-style paravirtualized VM will not see a hw clock. This should not cause false alarms on boot.

console

There should be a working console entry for the console device the kernel uses (hvc0, xvc0) Xen can also show a VNC console instead of the "serial" xen console. A nice setup gives the user a console in both places.

xenstore

The xenstore is a way for userland applications to exchange data between dom0 and domU. The utilties are called xenstore-* and they should be installed separately of the full xen toolset. Otherwise people need to build their own packages for it.

scheduling

Make sure the Xen version you package for dom0 has full power management support. (Can be tested using xenpm)

CPU hotplug

Xen can hotplug and remove additional vcpus to a VM. The domU kernel has to notice, and make use of these resources. The mechanism to online and offline them is standard linux, but often they are not noticed by the domU kernel.

Memory hotplug

Xen can hotplug and hot-remove memory to a VM. Make sure your guest kernel notices it when it's given another 16GB of ram :)

TMEM / Ramster

This item only applies to linux kernel 3.4 and above. The advanced memory management allows free memory to be pooled between virtual machines, duplicate pages to be de-duplicated and even adds memory compression. If your distro packages a 3.4 kernel, then the drivers should be included. (The patches can be applied even to 2.6.18+, but are not included as standard.)

Networking

The paravirtualized network drivers need to work and be enabled by default. Test if a MTU > 1500 bytes is working in domU Test if ethtool correctly shows CRC TX / RX offload as disabled. Otherwise no CRC verification is taking place in the Linux kernel which may cause data corruption. Some distro used to have RARP issues with Xen, see if it is working. (can be tested using arping) Test that the paravirtualized driver is always included in the ramdisks and your installer. People may need it i.e. for PXE installs.

Disk

Make sure the PV drivers working and are activated in the default OS install and also in the install Ramdisk. Otherwise the user will have a very slow and unhappy experience. Disks are of course also hot-pluggable and hot-removable. This should be working to avoid unneeded downtime (i.e. with NPIV on Linux this is very much required)

Xen Eventchannels

The Xen evenchannel allows dom0 to communitcate with the domU kernel. It can for example trigger a sysrq to the DomU to make it sync it's disks or panic. It can also ask the domU to run a shutdown much like the power button trigger does in ACPI. These features have to be enabled and also need to work during OS install, otherwise it's not possible to shutdown a VM that hung during install. (Test is easy - xl shutdown domUname)

Snapshotting

Running a snapshot should be possible

Cold migration

A cold migration of a VM between dom0's should be possible

Live migration

A live migration of a VM between dom0's should be possible. The clock should stay OK...

PCI

It should be possible to delegate a PCI device. Especially, don't strip down the domU kernel too much, it should still support PCI :)