Difference between revisions of "XAPI toolstack on a Debian-based distribution"

From Xen
(Created page with "__NOTOC__ = Installing the XCP toolstack on a Debian-based distribution = This is Project Kronos. Follow these instructions to get xapi, the XenAPI server daemon, installed on …")
 
 
(73 intermediate revisions by 15 users not shown)
Line 1: Line 1:
__NOTOC__
+
{{Needs_Review|This page is out-of-date. Many of the issues described on this page have been resolved.}}
= Installing the XCP toolstack on a Debian-based distribution =
 
  
 +
This is part of [[Project Kronos]]. Follow these instructions to get XAPI toolstack (including xapi, the XenAPI server daemon) installed on an Ubuntu Oneiric or Debian Squeeze box, using the upstream Linux kernel. This will effectively allow you to turn an existing Ubuntu or Debian machine into an XCP host. Note that this is *very much* a work-in-progress. Some features will be broken, and there will be a bit of manual configuration required. We appreciate any testing you do for this project, but please don't consider this production ready yet! Please visit the [[Kronos Supported Features]] page for a list of XCP features that are known to work on Kronos. The debian packages can be compiled from source following the instructions linked from the [[Project Kronos]] page.
  
This is Project Kronos. Follow these instructions to get xapi, the XenAPI server daemon, installed on an Ubuntu Oneiric or Debian Squeeze box, using the upstream Linux kernel. This will effectively allow you to turn an existing Ubuntu or Debian machine into an XCP host. Note that this is *very much* a work-in-progress. Some features will be broken, and there will be a bit of manual configuration required. We appreciate any testing you do for this project, but please don't consider this production ready yet! Please visit the [[Kronos Supported Features]] page for a list of XCP features that are known to work on Kronos.
+
=Note for Debian users=
  
__TOC__
+
If you intend to use Debian, we recommend you to carefully read /usr/share/doc/xcp-xapi/README.Debian.gz, which has notes on how to use XCP/Kronos in Debian, and which is maintained to be up-to-date with latest development. The Debian maintainer will update it, but it's not their job to update this wiki (which I just did still... :) ).
  
===What's the best environment?===
+
=What's the best environment?=
The best environment is a clean installed modern system with significant amounts of RAM in it and several cores.  However, this guide is written in the knowledge that not everyone will be in the lucky position of having such a box to play with.  In fact, many of us will be installing it on a moderately-sized box with an existing (upgraded) version of Ubuntu running.  One of the authors of this guide (MikeCamel) is in exactly that position, so you are not alone.  Hopefully this guide will be relevant to you, and will provide working instructions.  If you have a problem, let us know.
+
The best environment is a clean installed modern system with significant amounts of RAM and several cores.  However, this guide is written in the knowledge that not everyone will be in the lucky position of having such a box to play with.  In fact, many of us will be installing it on a moderately-sized box with an existing (upgraded) version of Ubuntu or Debian running.  One of the authors of this guide ([[User:MikeCamel|MikeCamel]]) is in exactly that position, so you are not alone.  Hopefully this guide will be relevant to you, and will provide working instructions.  If you have a problem, let us know.
  
==Warnings - README==
+
'''A note about kernels'''
You need to read these warningsIf you don't, you could do serious damage to your system, leave it unbootable, be surprised when things don't work as you expect (or at all), and generally fail to create World Peace[tm]And that would be a pity.
+
Kronos is really aimed at distributions (such as Ubuntu Precise Pangolin or Debian SID/Wheezy) which use the 3.2 kernelThere is a nasty bug in the 3.0 series (and probably before) which means that, between them, a couple of the components of Xen are going to reduce the size of Dom0 (and therefore your main operating environment) to something under 1 GiB, which may cause you significant problems. There are work-arounds, but this is a real issue if you're planning to run the XAPI toolstack on Oneiric, for instanceThere is a [[Using_XCP_-_preparing_the_toolstack|work-around]] available.
  
'''An important note''': don't be tempted to install any kernel (via apt, for instance) which ends in "-virtual" during this process. Such kernels are intended to run as Guests (that is, "VMs"), and are not configured to be used as the Dom0 kernel. You will run into particular problems if you try this on a system which uses an Nvidia card, as the nvidia-current drivers do not currently seem to compile against these kernels, and you will therefore not be able to start X correctly. Given the standard Oneiric start up and splash screens, this isn't a particularly easy problem to fix, so avoid it in the first place!
+
= Prepare your system =
  
'''Another important note''': the instructions on this page assume that:
+
{| class="prettytable" align="left" align="top"
*you know what you are doing with Linux, and are able to repair any issues that might arise as a result of following (or mis-following...) the instructions;
 
*you are running as root. If you don't know what this entails, then see 1. above, and seek further help from the community before proceeding.
 
 
 
'''A couple of network-related notes'''
 
# XAPI expects to run in a clean environment, and listens on ports 80 and 443.  If you're running anything else on these ports (typically Apache/Apache2), then you're likely to have problems.
 
# There is a known bug (which needs to be fixed) which means that in certain circumstances when you are using VPN software (e.g. when you have a tun0 interface listed in ifconfig), xapi will fail.  Please ensure that you remove all such software before attempting to run xcp-xapi.
 
 
 
'''And a note about kernels'''
 
Kronos is really aimed at Precise Pangolin (Ubuntu 12.04), which will include the 3.2 kernel.  There is a nasty bug in the 3.0 series (and probably before) which means that, between them, a couple of the components of Xen are going to reduce the size of Dom0 (and therefore your main operating environment) to something under 1 GiB, which may cause you significant problems. There are work-arounds (TODO: provide work-arounds...), but this is a real issue if you're planning to run Kronos on Oneiric.  There is a [[#ballooning-work-around|work-around described below]].
 
 
 
{| border="1" align="left" align="top"
 
 
|-
 
|-
 
!style="width: 10%" | Action
 
!style="width: 10%" | Action
! style="width: 45%" | Debian Squeeze
+
! style="width: 30%" | Debian SID / Wheezy
! style="width: 45%" | Ubuntu Oneiric
+
! style="width: 30%" | Ubuntu Oneiric or Precise
 
|-
 
|-
 
| Install OS
 
| Install OS
Line 49: Line 38:
 
</pre>
 
</pre>
  
| Install Ubuntu Oneiric.
+
| Install Ubuntu Oneiric or Precise.
Kronos will work with either Ubuntu Oneiric Server or Desktop, with the default kernel. If you want to use local storage, you will have to reserve an unformatted partition on your hard disk for xapi. In a later step, you will use this partition to create a local storage repository.
+
Kronos will work with either Ubuntu Precise Server or Desktop, with the default kernel. If you want to use local storage, you will have to reserve an unformatted partition on your hard disk for xapi. In a later step, you will use this partition to create a local storage repository.  Note that the xcp-xapi packages are in Precise, but may have dependency errors.  To use these, omit the "Set up repositories" step for Precise.
 
|-  
 
|-  
 
| Install any required firmware
 
| Install any required firmware
 
| Install firmware (only if necessary - http://wiki.debian.org/Firmware)
 
| Install firmware (only if necessary - http://wiki.debian.org/Firmware)
|  
+
|
 
|-
 
|-
 
| Set up repositories
 
| Set up repositories
| Add the following to /etc/apt/sources.list
+
| This part isn't needed anymore in Debian, as XCP is now in SID.
 +
| For Oneiric: add the following to /etc/apt/sources.list
 
<pre>
 
<pre>
deb http://ftp.uk.debian.org/debian/ experimental main
+
deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main
deb-src http://ftp.uk.debian.org/debian/ experimental main
+
deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main
 
</pre>
 
</pre>
| Add the following to /etc/apt/sources.list
+
run this (with sudo):
 
<pre>
 
<pre>
deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main
+
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9273A937
deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main
 
 
</pre>
 
</pre>
|}
 
  
==Install xapi==
+
or
 +
 
 +
For Precise: add the following to /etc/apt/sources.list
 +
<pre>
 +
deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu precise main
 +
deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu precise main
 +
</pre>
 +
run this (with sudo):
 +
<pre>
 +
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9273A937
 +
</pre>
  
This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.
 
  
 +
or (for Oneiric or Precise)
  
 
<pre>
 
<pre>
apt-get update
+
apt-get install python-software-properties
apt-get install xcp-xapi
+
add-apt-repository ppa:ubuntu-xen-org/xcp-unstable
apt-get install xcp-xe
 
 
</pre>
 
</pre>
  
 +
|}
  
Once you've installed these packages, you can just do an `apt-get update && apt-get upgrade` to keep up with the latest builds.
+
=Work around Open vSwitch bug (Ubuntu Oneiric)=
 +
 
 +
If you are installing onto a fresh Oneiric system, then you will hit https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/894754.  This is fixed in Precise. The following steps also resolve it on that setup: To work around this:
  
==Post-install setup==
+
<pre>
 +
apt-get update
 +
apt-get install openvswitch-datapath-dkms
 +
</pre>
  
'''a)''' Optional: Make xen the default grub entry.
+
=Workaround xcp-networkd missing file (Ubuntu)=
 +
Before installing xapi run the following commands to work around a missing file in the packages:
  
 
<pre>
 
<pre>
sed -i 's/GRUB_DEFAULT=""/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
+
mkdir /etc/xcp
update-grub
+
echo "bridge" >> /etc/xcp/network.conf
 
</pre>
 
</pre>
  
'''Note:''' given the default set up for Ubuntu's grub (in /etc/default/grub), you don't get to see the splashscreen with kernel boot options. If you do want to see the list of options, you may need to change some of the settings in this file.
+
This will be addressed properly in a future release
  
'''b.''' Setup the network/interfaces file:
+
Note that this is already fixed in Debian SID / Wheezy, with xcp-networkd asking you (through the Debconf interface) which type of network configuration you wish to use.
  
This is likely to be /etc/network interfaces - ensure that you don't remove any existing settings.  Note that if you are running a DHCP server on this machine, then you may need to change the xenbr0 interface to a static or manual IP address, as DHCP may not come up in time to provide an address otherwise.
+
=Install xapi (Ubuntu)=
  
<pre>
+
This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.
auto lo xenbr0
 
 
 
iface xenbr0 inet dhcp
 
        bridge_ports eth0
 
</pre>
 
  
'''c.''' Setup the xensource-inventory file:
+
Note: XCP requires lspci, part of the pciutils package. If you are installing on top of a freshly debootstrap'd install, you may not have this available, <pre> apt-get install pciutils </pre> will resolve this, otherwise xcp-xapi will fail to start and give you a connection refused error.
  
'''Note:''' this assumes that the root partition is on /dev/sda1 - change if required.
 
'''Note:''' this assumes that your management interface will be xenbr0 - change if required.
 
  
 
<pre>
 
<pre>
control_domain=`uuidgen`
+
apt-get update
installation=`uuidgen`
+
apt-get install xcp-xapi
 
+
apt-get install xcp-xe
cat > /etc/xcp/xensource-inventory << EOF
 
CURRENT_INTERFACES='xenbr0'
 
BUILD_NUMBER='0'
 
CONTROL_DOMAIN_UUID='${control_domain}'
 
INSTALLATION_UUID='${installation}'
 
MANAGEMENT_INTERFACE='xenbr0'
 
PRIMARY_DISK='/dev/sda1'
 
EOF
 
 
</pre>
 
</pre>
  
==Reboot.==
 
  
= Using XCP =
+
Once you've installed these packages, you can just do an `apt-get update && apt-get upgrade` to keep up with the latest builds.
  
Ensure that xapi is running with "service xapi status". If you don't get the response "* xapi is running", you will need to run "service xapi start".
+
=Workaround VM's not going to power-state: halted after shutdown (Ubuntu)=
  
Set up your path:
+
In Ubuntu (eg: not in Debian SID / Wheezy) the current version of the packages, there is a bug that will prevent a VM from going to a halted power-state when you shut them down. This is caused by the VIF's not being properly torn down after the shutdown.  This will be addressed in a future release.  You can address this issue by editing the file at /usr/lib/xcp/scripts/vif.  Go to the very end of the file and find the block of code below (around line 258):
  
 
<pre>
 
<pre>
export PATH=$PATH:/usr/lib/xen-common/xapi/bin
+
remove)
 +
    if [ "${TYPE}" = "vif" ] ;then
 +
        call_hook_script $DOMID "${ACTION}"
 +
                # Unclear whether this is necessary, since netback also does it:
 +
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
 +
        xenstore-rm "${HOTPLUG_STATUS}"
 
</pre>
 
</pre>
  
 
+
and change the case from remove) to remove|offline) so that it looks like this:
Set up a storage repository. For now, we recommend NFS, or EXT if you have an unused block device. For example:
 
  
 
<pre>
 
<pre>
SR=`xe sr-create type=nfs name-label=nfs device-config:server=<nfs server> device-config:serverpath=<path on server>`
+
remove|offline)
 +
    if [ "${TYPE}" = "vif" ] ;then
 +
        call_hook_script $DOMID "${ACTION}"
 +
                # Unclear whether this is necessary, since netback also does it:
 +
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
 +
        xenstore-rm "${HOTPLUG_STATUS}"
 
</pre>
 
</pre>
  
or
+
=Workaround XAPI conflicts with XEND (Debian and Ubuntu)=
  
'''Note: DANGER!!!''' The partition selected will be formatted, and all data on it wiped. Ensure that you select the device correctly and that you have backed up any existing information on this partition.
+
You will need to disable xend from starting in order to get xcp-xapi to start.  This is done by modifying the file at /etc/init.d/xend. Many of the setup processes in this script still need to run, but xend itself should not start.  This will be resolved in a future release. The command to resolve this is:
  
 
<pre>
 
<pre>
SR=`xe sr-create type=ext device-config:device=/dev/sda3 name-label=ext`
+
sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend
 +
update-rc.d xendomains disable
 
</pre>
 
</pre>
  
Set this as pool default SR:
+
=Workaround qemu keymap location preventing vncterm from starting=
 +
 
 +
In order for vncterm to start up with a Linux VM, it needs to load the qemu keymaps however they are located in a location other than where it looks, this can be resolved with the following commands
 +
 
 +
mkdir /usr/share/qemu
 +
ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps
 +
 
 +
=Post-install setup=
 +
 
 +
==Optional: Make xen the default grub entry.==
  
 
<pre>
 
<pre>
POOL=`xe pool-list --minimal`
+
sed -i 's/GRUB_DEFAULT=.\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
xe pool-param-set uuid=$POOL default-SR=$SR
+
update-grub
 
</pre>
 
</pre>
  
==<div id="ballooning-work-around">Memory and Dom0</div>==
+
'''Note:''' given the default set up for Ubuntu's grub (in /etc/default/grub), you don't get to see the splashscreen with kernel boot options. If you do want to see the list of options, you may need to change some of the settings in this file.
There is a bug within the 3.0 series kernel that is shipped with Oneiric, but is fixed in the 3.2 kernel which should be shipped with Precious, where mis-reporting of memory leads xapi and squeezed to reduce (sometimes significantly!) the amount of memory available to Dom0.
+
 
 +
Another way to do this, as advised by the Debian wiki and in the Debian release notes is:
 +
 
 +
<pre>cd /etc/grub.d
 +
mv 10_linux 25_linux
 +
update-grub2</pre>
  
''Note'': certain versions of the package which installed squeezed failed to set the relevant scripts to start it at boot.  Ensure that you have the latest release, and that you have checked that squeezed is running -
+
==Setup the network/interfaces file==
<pre>ps ax | grep sqeezed</pre>
 
- before completing the steps below.
 
  
Your Dom0 should never be set to consume all of your physical memory (otherwise - where would guest VMs run?), but if, on initially running xapi, you notice that performance has suddenly dropped significantly, check the amount of memory available via <pre>cat /proc/meminfo</pre> (MemTotal, quoted in bytes). If this seems ridiculously low, then you will need to set hard parameters for the amount of memory assigned to Dom0. First you need to find the uuid of Dom0 thus:
+
This is likely to be /etc/network/interfaces - ensure that you don't remove any existing settings. Note that if you are running a DHCP server on this machine, then you may need to change the xenbr0 interface to a static or manual IP address, as DHCP may not come up in time to provide an address otherwise.
  
 
<pre>
 
<pre>
xe vm-list
+
auto lo xenbr0
 +
 
 +
iface xenbr0 inet dhcp
 +
        bridge_ports eth0
 
</pre>
 
</pre>
  
The first entry should look at little like this:
+
If you don't use dhcp, but a fixed IP, then you can do:
<pre>
 
uuid ( RO)          : f5d0039b-1138-4635-c153-6203bfdc330f
 
    name-label ( RW): Control domain on host: piggy
 
    power-state ( RO): running
 
</pre>
 
We will use this as an example.
 
  
Next, you need to decide how much memory to give Dom0.  In our example, we are going to give it 2 GiB.  Run the following commands, substituting the correct uuid and memory parameters:
+
<pre>auto lo
<pre>
+
iface lo inet loopback
xe vm-param-set uuid=f5d0039b-1138-4635-c153-6203bfdc330f memory-dynamic-max=2GiB
 
xe vm-param-set uuid=f5d0039b-1138-4635-c153-6203bfdc330f memory-dynamic-min=2GiB
 
</pre>
 
Note that this will work around the problem, but will not allow Xen to adjust the amount of memory available to Dom0 automatically.  Reboot your machine (start squeezed manually if required), and check meminfo again.  Note that you will not see the entire amount of memory assigned - some will be used - but you can now balance this correctly.
 
  
 +
# The primary network interface
 +
auto xenbr0
 +
iface xenbr0 inet static
 +
bridge_ports eth0
 +
address 1.2.3.4
 +
netmask 255.255.255.0
 +
network 1.2.3.0
 +
broadcast 1.2.3.255
 +
gateway 1.2.3.1</pre>
  
==Install a Guest==
+
Also, if you use the openvswitch mode (and not bridging), then you will need to tell XCP about your network configuration:
  
Now that you have XCP set up, you probably want to install some guests. We provide examples of how to set up a Windows guest or a Linux guest. You can, of course, mix and match.
+
<pre>PIF_UUID=`xe pif-list device=eth0 --minimal`
* [[Installing Linux on Kronos]]
+
xe pif-reconfigure-ip uuid=$PIF_UUID mode=static IP=1.2.3.4 netmask=255.255.255.0 gateway=1.2.3.1</pre>
* [[Installing Windows on Kronos]]
 
  
 +
or for DHCP:
  
= Compiling from source =
+
<pre>PIF_UUID=`xe pif-list device=eth0 --minimal`
 +
xe pif-reconfigure-ip uuid=$PIF_UUID mode=dhcp</pre>
  
The easiest way to compile the XCP toolstack from source is to use the [https://github.com/jonludlam/xapi-autobuilder xapi-autobuilder].
+
This is very important, as otherwise, you might loose network connectivity. I believe (I didn't test) that if you don't have physical access to the server, the correct way to do this is to first use the bridging mode, then do the above pif-reconfigure-ip, then switch to openvswitch mode and reboot.
  
First install the perquisites:
+
==Setup the xensource-inventory file==
  
 +
'''Note:''' this assumes that the root partition is on /dev/sda1 - change if required.
 +
'''Note:''' this assumes that your management interface will be xenbr0 - change if required.
 +
'''Note:''' In order to increase compatibility with XenCenter/OpenStack etc you can add the line 'PRODUCT_VERSION=6.0.0' to the /etc/xcp/inventory file.
 +
'''Note:''' You normally don't need this in Debian SID (I'm not sure for Ubuntu) as everything will be setup automatically. The PRIMARY_DISK for example is not needed. The PRODUCT_VERSION thing above is however needed if you intend to use Openstack.
  
 
<pre>
 
<pre>
apt-get install pbuilder debhelper dh-ocaml dh-autoreconf cdebootstrap python-debian mercurial git
+
control_domain=`uuidgen`
wget http://downloads.xen.org/XCP/debian/blktap-dkms_0.1_all.deb
+
installation=`uuidgen`
dpkg -i blktap-dkms_0.1_all.deb
+
 
 +
cat > /etc/xcp/inventory << EOF
 +
CURRENT_INTERFACES='xenbr0'
 +
BUILD_NUMBER='0'
 +
CONTROL_DOMAIN_UUID='${control_domain}'
 +
INSTALLATION_UUID='${installation}'
 +
MANAGEMENT_INTERFACE='xenbr0'
 +
PRIMARY_DISK='/dev/sda1'
 +
EOF
 
</pre>
 
</pre>
  
 +
==Set the Xen toolstack==
  
Then clone xapi-autobuilder and build:
+
You will need to set a TOOLSTACK variable to get xcp-xapi to start, you can do that with the following:
 
 
  
 
<pre>
 
<pre>
git clone https://github.com/jonludlam/xapi-autobuilder.git
+
echo 'TOOLSTACK="xapi"' > /etc/default/xen
cd xapi-autobuilder
 
make clean ; make
 
 
</pre>
 
</pre>
  
 +
== Also See ==
 +
=== Debian ===
 +
* [http://packages.debian.org/wheezy/xcp-xapi xcp-xapi packages in Debian]
 +
* [http://wiki.debian.org/XCP XCP wiki in Debian]
 +
=== Ubuntu ===
 +
* [http://packages.ubuntu.com/precise/xcp-xapi xcp-xapi packages in Ubuntu]
 +
* [https://help.ubuntu.com/community/XenProposed Xen Documentation on Ubuntu]
 +
* [http://manpages.ubuntu.com/manpages/precise/man1/xapi.1.html XCP Documentation on Ubuntu]
 +
* [[:Category:OpenStack|XCP-OpenStack Documentation]]
  
This will build all of the required Debian packages for Xapi (except for vncterm, which is only available as a binary until we sort out licensing issues). Take a look at xapi-autobuilder/build.sh to see what's going on inside. You can modify the files git-repos and hg-repos to point to your own repositories if you plan on compiling your own code. If you only want to build a subset of the packages, comment out the ones you don't want to build from the build.sh script.
+
=Reboot=
  
You can manually download all of the latest packages and source packages using:
 
  
  
<pre>
+
[[Category:XAPI Users]]
wget -r -l1 --no-parent -nd http://downloads.xen.org/XCP/debian/latest/
+
[[Category:HowTo]]
rm index.html*
 
</pre>
 
  
[[Category:XCP]]
+
[[Category:Debian]]
[[Category:HowTo]]
+
[[Category:Ubuntu]]
[[Category:Design Document]]
+
[[Category:OpenStack]]
[[Category:Developers]]
+
[[Category:Host Install]]
[[Category:Users]]
 

Latest revision as of 14:10, 11 July 2013

Icon todo.png Needs Review

This page is out-of-date. Many of the issues described on this page have been resolved.


This is part of Project Kronos. Follow these instructions to get XAPI toolstack (including xapi, the XenAPI server daemon) installed on an Ubuntu Oneiric or Debian Squeeze box, using the upstream Linux kernel. This will effectively allow you to turn an existing Ubuntu or Debian machine into an XCP host. Note that this is *very much* a work-in-progress. Some features will be broken, and there will be a bit of manual configuration required. We appreciate any testing you do for this project, but please don't consider this production ready yet! Please visit the Kronos Supported Features page for a list of XCP features that are known to work on Kronos. The debian packages can be compiled from source following the instructions linked from the Project Kronos page.

Note for Debian users

If you intend to use Debian, we recommend you to carefully read /usr/share/doc/xcp-xapi/README.Debian.gz, which has notes on how to use XCP/Kronos in Debian, and which is maintained to be up-to-date with latest development. The Debian maintainer will update it, but it's not their job to update this wiki (which I just did still... :) ).

What's the best environment?

The best environment is a clean installed modern system with significant amounts of RAM and several cores. However, this guide is written in the knowledge that not everyone will be in the lucky position of having such a box to play with. In fact, many of us will be installing it on a moderately-sized box with an existing (upgraded) version of Ubuntu or Debian running. One of the authors of this guide (MikeCamel) is in exactly that position, so you are not alone. Hopefully this guide will be relevant to you, and will provide working instructions. If you have a problem, let us know.

A note about kernels Kronos is really aimed at distributions (such as Ubuntu Precise Pangolin or Debian SID/Wheezy) which use the 3.2 kernel. There is a nasty bug in the 3.0 series (and probably before) which means that, between them, a couple of the components of Xen are going to reduce the size of Dom0 (and therefore your main operating environment) to something under 1 GiB, which may cause you significant problems. There are work-arounds, but this is a real issue if you're planning to run the XAPI toolstack on Oneiric, for instance. There is a work-around available.

Prepare your system

Action Debian SID / Wheezy Ubuntu Oneiric or Precise
Install OS Install Debian Squeeze.

Convert to Unstable:

  • Edit /etc/apt/sources.list and replace 'squeeze' with 'unstable'
perl -pi -e 's/squeeze/unstable/g' /etc/apt/sources.list
apt-get update
apt-get dist-upgrade
Install Ubuntu Oneiric or Precise.

Kronos will work with either Ubuntu Precise Server or Desktop, with the default kernel. If you want to use local storage, you will have to reserve an unformatted partition on your hard disk for xapi. In a later step, you will use this partition to create a local storage repository. Note that the xcp-xapi packages are in Precise, but may have dependency errors. To use these, omit the "Set up repositories" step for Precise.

Install any required firmware Install firmware (only if necessary - http://wiki.debian.org/Firmware)
Set up repositories This part isn't needed anymore in Debian, as XCP is now in SID. For Oneiric: add the following to /etc/apt/sources.list
deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main
deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu oneiric main

run this (with sudo):

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9273A937

or

For Precise: add the following to /etc/apt/sources.list

deb http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu precise main
deb-src http://ppa.launchpad.net/ubuntu-xen-org/xcp-unstable/ubuntu precise main

run this (with sudo):

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9273A937


or (for Oneiric or Precise)

apt-get install python-software-properties
add-apt-repository ppa:ubuntu-xen-org/xcp-unstable

Work around Open vSwitch bug (Ubuntu Oneiric)

If you are installing onto a fresh Oneiric system, then you will hit https://bugs.launchpad.net/ubuntu/+source/openvswitch/+bug/894754. This is fixed in Precise. The following steps also resolve it on that setup: To work around this:

apt-get update
apt-get install openvswitch-datapath-dkms

Workaround xcp-networkd missing file (Ubuntu)

Before installing xapi run the following commands to work around a missing file in the packages:

mkdir /etc/xcp
echo "bridge" >> /etc/xcp/network.conf

This will be addressed properly in a future release

Note that this is already fixed in Debian SID / Wheezy, with xcp-networkd asking you (through the Debconf interface) which type of network configuration you wish to use.

Install xapi (Ubuntu)

This step installs XCP's xapi and all its dependencies, including the Xen hypervisor.

Note: XCP requires lspci, part of the pciutils package. If you are installing on top of a freshly debootstrap'd install, you may not have this available,

 apt-get install pciutils 

will resolve this, otherwise xcp-xapi will fail to start and give you a connection refused error.


apt-get update
apt-get install xcp-xapi
apt-get install xcp-xe


Once you've installed these packages, you can just do an `apt-get update && apt-get upgrade` to keep up with the latest builds.

Workaround VM's not going to power-state: halted after shutdown (Ubuntu)

In Ubuntu (eg: not in Debian SID / Wheezy) the current version of the packages, there is a bug that will prevent a VM from going to a halted power-state when you shut them down. This is caused by the VIF's not being properly torn down after the shutdown. This will be addressed in a future release. You can address this issue by editing the file at /usr/lib/xcp/scripts/vif. Go to the very end of the file and find the block of code below (around line 258):

remove)
    if [ "${TYPE}" = "vif" ] ;then
        call_hook_script $DOMID "${ACTION}"
                # Unclear whether this is necessary, since netback also does it:
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
        xenstore-rm "${HOTPLUG_STATUS}"

and change the case from remove) to remove|offline) so that it looks like this:

remove|offline)
    if [ "${TYPE}" = "vif" ] ;then
        call_hook_script $DOMID "${ACTION}"
                # Unclear whether this is necessary, since netback also does it:
        logger -t script-vif "${dev}: removing ${HOTPLUG_STATUS}"
        xenstore-rm "${HOTPLUG_STATUS}"

Workaround XAPI conflicts with XEND (Debian and Ubuntu)

You will need to disable xend from starting in order to get xcp-xapi to start. This is done by modifying the file at /etc/init.d/xend. Many of the setup processes in this script still need to run, but xend itself should not start. This will be resolved in a future release. The command to resolve this is:

sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend
update-rc.d xendomains disable

Workaround qemu keymap location preventing vncterm from starting

In order for vncterm to start up with a Linux VM, it needs to load the qemu keymaps however they are located in a location other than where it looks, this can be resolved with the following commands

mkdir /usr/share/qemu
ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps

Post-install setup

Optional: Make xen the default grub entry.

sed -i 's/GRUB_DEFAULT=.\+/GRUB_DEFAULT="Xen 4.1-amd64"/' /etc/default/grub
update-grub

Note: given the default set up for Ubuntu's grub (in /etc/default/grub), you don't get to see the splashscreen with kernel boot options. If you do want to see the list of options, you may need to change some of the settings in this file.

Another way to do this, as advised by the Debian wiki and in the Debian release notes is:

cd /etc/grub.d
mv 10_linux 25_linux
update-grub2

Setup the network/interfaces file

This is likely to be /etc/network/interfaces - ensure that you don't remove any existing settings. Note that if you are running a DHCP server on this machine, then you may need to change the xenbr0 interface to a static or manual IP address, as DHCP may not come up in time to provide an address otherwise.

auto lo xenbr0

iface xenbr0 inet dhcp
        bridge_ports eth0 

If you don't use dhcp, but a fixed IP, then you can do:

auto lo
iface lo inet loopback

# The primary network interface
auto xenbr0
iface xenbr0 inet static
	bridge_ports eth0
	address 1.2.3.4
	netmask 255.255.255.0
	network 1.2.3.0
	broadcast 1.2.3.255
	gateway 1.2.3.1

Also, if you use the openvswitch mode (and not bridging), then you will need to tell XCP about your network configuration:

PIF_UUID=`xe pif-list device=eth0 --minimal`
xe pif-reconfigure-ip uuid=$PIF_UUID mode=static IP=1.2.3.4 netmask=255.255.255.0 gateway=1.2.3.1

or for DHCP:

PIF_UUID=`xe pif-list device=eth0 --minimal`
xe pif-reconfigure-ip uuid=$PIF_UUID mode=dhcp

This is very important, as otherwise, you might loose network connectivity. I believe (I didn't test) that if you don't have physical access to the server, the correct way to do this is to first use the bridging mode, then do the above pif-reconfigure-ip, then switch to openvswitch mode and reboot.

Setup the xensource-inventory file

Note: this assumes that the root partition is on /dev/sda1 - change if required. Note: this assumes that your management interface will be xenbr0 - change if required. Note: In order to increase compatibility with XenCenter/OpenStack etc you can add the line 'PRODUCT_VERSION=6.0.0' to the /etc/xcp/inventory file. Note: You normally don't need this in Debian SID (I'm not sure for Ubuntu) as everything will be setup automatically. The PRIMARY_DISK for example is not needed. The PRODUCT_VERSION thing above is however needed if you intend to use Openstack.

control_domain=`uuidgen`
installation=`uuidgen`

cat > /etc/xcp/inventory << EOF
CURRENT_INTERFACES='xenbr0'
BUILD_NUMBER='0'
CONTROL_DOMAIN_UUID='${control_domain}' 
INSTALLATION_UUID='${installation}'
MANAGEMENT_INTERFACE='xenbr0'
PRIMARY_DISK='/dev/sda1'
EOF

Set the Xen toolstack

You will need to set a TOOLSTACK variable to get xcp-xapi to start, you can do that with the following:

echo 'TOOLSTACK="xapi"' > /etc/default/xen

Also See

Debian

Ubuntu

Reboot