XAPI toolstack on a Debian-based distribution
Installing the XCP toolstack on a Debian-based distribution
This is Project Kronos. Follow these instructions to get xapi, the XenAPI server daemon, installed on an Ubuntu Oneiric or Debian Squeeze box, using the upstream Linux kernel. This will effectively allow you to turn an existing Ubuntu or Debian machine into an XCP host. Note that this is *very much* a work-in-progress. Some features will be broken, and there will be a bit of manual configuration required. We appreciate any testing you do for this project, but please don't consider this production ready yet! Please visit the Kronos Supported Features page for a list of XCP features that are known to work on Kronos.
What's the best environment?
The best environment is a clean installed modern system with significant amounts of RAM in it and several cores. However, this guide is written in the knowledge that not everyone will be in the lucky position of having such a box to play with. In fact, many of us will be installing it on a moderately-sized box with an existing (upgraded) version of Ubuntu or Debian running. One of the authors of this guide (MikeCamel) is in exactly that position, so you are not alone. Hopefully this guide will be relevant to you, and will provide working instructions. If you have a problem, let us know.
A note about kernels Kronos is really aimed at distributions (such as Ubuntu Precise Pangolin) which use the 3.2 kernel. There is a nasty bug in the 3.0 series (and probably before) which means that, between them, a couple of the components of Xen are going to reduce the size of Dom0 (and therefore your main operating environment) to something under 1 GiB, which may cause you significant problems. There are work-arounds (TODO: provide work-arounds...), but this is a real issue if you're planning to run the XCP toolstack on Oneiric, for instance. There is a work-around described below.
| Action | Debian Squeeze | Ubuntu Oneiric |
|---|---|---|
| Install OS | Install Debian Squeeze.
Convert to Unstable:
perl -pi -e 's/squeeze/unstable/g' /etc/apt/sources.list apt-get update apt-get dist-upgrade |
Install Ubuntu Oneiric.
Kronos will work with either Ubuntu Oneiric Server or Desktop, with the default kernel. If you want to use local storage, you will have to reserve an unformatted partition on your hard disk for xapi. In a later step, you will use this partition to create a local storage repository. |
| Install any required firmware | Install firmware (only if necessary - http://wiki.debian.org/Firmware) | |
| Set up repositories | Add the following to /etc/apt/sources.list
deb http://ftp.uk.debian.org/debian/ experimental main deb-src http://ftp.uk.debian.org/debian/ experimental main |
Add the following to /etc/apt/sources.list
deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main |
Install xapi
This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.
apt-get update apt-get install xcp-xapi apt-get install xcp-xe
Once you've installed these packages, you can just do an `apt-get update && apt-get upgrade` to keep up with the latest builds.
Post-install setup
a) Optional: Make xen the default grub entry.
sed -i 's/GRUB_DEFAULT=""/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub update-grub
Note: given the default set up for Ubuntu's grub (in /etc/default/grub), you don't get to see the splashscreen with kernel boot options. If you do want to see the list of options, you may need to change some of the settings in this file.
b. Setup the network/interfaces file:
This is likely to be /etc/network interfaces - ensure that you don't remove any existing settings. Note that if you are running a DHCP server on this machine, then you may need to change the xenbr0 interface to a static or manual IP address, as DHCP may not come up in time to provide an address otherwise.
auto lo xenbr0
iface xenbr0 inet dhcp
bridge_ports eth0
c. Setup the xensource-inventory file:
Note: this assumes that the root partition is on /dev/sda1 - change if required. Note: this assumes that your management interface will be xenbr0 - change if required.
control_domain=`uuidgen`
installation=`uuidgen`
cat > /etc/xcp/xensource-inventory << EOF
CURRENT_INTERFACES='xenbr0'
BUILD_NUMBER='0'
CONTROL_DOMAIN_UUID='${control_domain}'
INSTALLATION_UUID='${installation}'
MANAGEMENT_INTERFACE='xenbr0'
PRIMARY_DISK='/dev/sda1'
EOF
Reboot.
Using XCP
Ensure that xapi is running with "service xapi status". If you don't get the response "* xapi is running", you will need to run "service xapi start".
Set up your path:
export PATH=$PATH:/usr/lib/xen-common/xapi/bin
Set up a storage repository. For now, we recommend NFS, or EXT if you have an unused block device. For example:
SR=`xe sr-create type=nfs name-label=nfs device-config:server=<nfs server> device-config:serverpath=<path on server>`
or
Note: DANGER!!! The partition selected will be formatted, and all data on it wiped. Ensure that you select the device correctly and that you have backed up any existing information on this partition.
SR=`xe sr-create type=ext device-config:device=/dev/sda3 name-label=ext`
Set this as pool default SR:
POOL=`xe pool-list --minimal` xe pool-param-set uuid=$POOL default-SR=$SR
Memory and Dom0
There is a bug within the 3.0 series kernel that is shipped with Oneiric, but is fixed in the 3.2 kernel which should be shipped with Precious, where mis-reporting of memory leads xapi and squeezed to reduce (sometimes significantly!) the amount of memory available to Dom0.
Note: certain versions of the package which installed squeezed failed to set the relevant scripts to start it at boot. Ensure that you have the latest release, and that you have checked that squeezed is running -
ps ax | grep sqeezed
- before completing the steps below.
Your Dom0 should never be set to consume all of your physical memory (otherwise - where would guest VMs run?), but if, on initially running xapi, you notice that performance has suddenly dropped significantly, check the amount of memory available via
cat /proc/meminfo
(MemTotal, quoted in bytes). If this seems ridiculously low, then you will need to set hard parameters for the amount of memory assigned to Dom0. First you need to find the uuid of Dom0 thus:
xe vm-list
The first entry should look at little like this:
uuid ( RO) : f5d0039b-1138-4635-c153-6203bfdc330f
name-label ( RW): Control domain on host: piggy
power-state ( RO): running
We will use this as an example.
Next, you need to decide how much memory to give Dom0. In our example, we are going to give it 2 GiB. Run the following commands, substituting the correct uuid and memory parameters:
xe vm-param-set uuid=f5d0039b-1138-4635-c153-6203bfdc330f memory-dynamic-max=2GiB xe vm-param-set uuid=f5d0039b-1138-4635-c153-6203bfdc330f memory-dynamic-min=2GiB
Note that this will work around the problem, but will not allow Xen to adjust the amount of memory available to Dom0 automatically. Reboot your machine (start squeezed manually if required), and check meminfo again. Note that you will not see the entire amount of memory assigned - some will be used - but you can now balance this correctly.
Install a Guest
Now that you have XCP set up, you probably want to install some guests. We provide examples of how to set up a Windows guest or a Linux guest. You can, of course, mix and match.
Compiling from source
The easiest way to compile the XCP toolstack from source is to use the xapi-autobuilder.
First install the perquisites:
apt-get install pbuilder debhelper dh-ocaml dh-autoreconf cdebootstrap python-debian mercurial git wget http://downloads.xen.org/XCP/debian/blktap-dkms_0.1_all.deb dpkg -i blktap-dkms_0.1_all.deb
Then clone xapi-autobuilder and build:
git clone https://github.com/jonludlam/xapi-autobuilder.git cd xapi-autobuilder make clean ; make
This will build all of the required Debian packages for Xapi (except for vncterm, which is only available as a binary until we sort out licensing issues). Take a look at xapi-autobuilder/build.sh to see what's going on inside. You can modify the files git-repos and hg-repos to point to your own repositories if you plan on compiling your own code. If you only want to build a subset of the packages, comment out the ones you don't want to build from the build.sh script.
You can manually download all of the latest packages and source packages using:
wget -r -l1 --no-parent -nd http://downloads.xen.org/XCP/debian/latest/ rm index.html*
