XCP Building Instructions
Building xapi on CentOS 6.3
In this guide we look at an experimental way to get xapi running on CentOS 6.3
This is not Zeus, there are no packages yet. This is a developer install using: http://opam.ocamlpro.com/
Setting up Xen
While not strictly required for the build, to help us test it, lets install Xen.
Minimal install of CentOS 6.3 x86_64. Use network install iso on VMware fusion, or full iso, minimal confuses things.
This is based on http://wiki.centos.org/QaWiki/Xen4
yum install wget cd /etc/yum.repos.d/ wget http://dev.centos.org/centos/6/xen-c6/xen-c6.repo yum repolist yum install kernel kernel-firmware yum install xen
Now need to edit /etc/grub.conf Note: check the paths and the kernel version, also check dom0 memory. Also: The above 3.4.x kernel should the second in the list, xen should be first.
title xen
root (hd0,0)
kernel /xen.gz dom0_mem=1024M,max:1024M loglvl=all guest_loglvl=all
module /vmlinuz-3.4.32-6.el6.centos.alt.x86_64 ro root=/dev/mapper/vg_jgarbuttcentos-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=uk LANG=en_US.UTF-8 rd_LVM_LV=vg_jgarbuttcentos/lv_root rd_NO_MD quiet SYSFONT=latarcyrheb-sun16 rhgb crashkernel=auto rd_NO_DM rd_LVM_LV=vg_jgarbuttcentos/lv_swap
module /initramfs-3.4.32-6.el6.centos.alt.x86_64.img
Reboot, then check all is well:
uname -a xl info
Install OCAML 3.12.1
Currently this is not part of CentOS 6.3
One way is to build the following Fedora 17 srpm: http://dl.fedoraproject.org/pub/fedora/linux/releases/17/Everything/source/SRPMS/o/ocaml-3.12.1-1.fc17.src.rpm
Instructions can be found here, including the required setup: http://wiki.centos.org/HowTos/RebuildSRPM
You can download some very rough packages, and install them like this:
mkdir ocaml cd ocaml wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-3.12.1-12.el6.x86_64.rpm wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-camlp4-3.12.1-12.el6.x86_64.rpm wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-camlp4-devel-3.12.1-12.el6.x86_64.rpm wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-runtime-3.12.1-12.el6.x86_64.rpm wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-docs-3.12.1-12.el6.x86_64.rpm wget http://8ac390279a4a48a6f8b1-aaca1c7c7029a65ba2e38949339e1d8b.r35.cf3.rackcdn.com/ocaml-ocamldoc-3.12.1-12.el6.x86_64.rpm yum install gdbm-devel ncurses-devel rpm-build rpm -i ocaml-*
The were created using this source rpm:
Install OPAM (from source)
Before you build, you will need:
- ocaml-3.12.1 ocaml-runtime ocaml-camlp4 ocaml-camlp4-dev (see above)
And a few other tools:
yum install make gcc
Build instructions are here: http://opam.ocamlpro.com/doc/Advanced_Install.html
Download the latest stable, and build it:
wget https://github.com/OCamlPro/opam/archive/latest.tar.gz -O opam-latest.tgz tar -xf opam-latest.tgz cd opam-latest ./configure make make install
Now you have OPAM you can initialise it:
opam init eval `opam config env`
To make sure you don't need the above line all the time, add the following into .bash_profile before the PATH is set:
eval `opam config env`
Install OCAML 4.00.1
It is now possible to install the latest OCAML
opam switch 4.00.1
Now setup the environment again:
eval `opam config env`
Install External Dependencies
Yum
There are a few dependencies we need:
yum install git xen-devel yum install libuuid-devel time pam-devel tk-devel libvirt-devel zlib-devel # you may not actually need tk-devel, try without to avoid X dependencies
Unpackaged
XCP has a custom version on vncterm:
wget http://a94cd2de16980073c274-9e5915cce229bfd373f03bf01a9a7c85.r57.cf3.rackcdn.com/vncterm-1.6.10-251.x86_64.rpm rpm -i vncterm-1.6.10-251.x86_64.rpm
This was built for CentOS 6.3 from the XCP srpm:
There is also an important python script:
wget http://a94cd2de16980073c274-9e5915cce229bfd373f03bf01a9a7c85.r57.cf3.rackcdn.com/eliloader -O /usr/bin/eliloader chmod +x /usr/bin/eliloader
This was taken from the debian version:
OPAM
There are quite a few XCP libraries we can install using OPAM:
opam install ocamlfind omake opam remote add xen-dev git://github.com/xen-org/opam-repo-dev opam install xen-api-libs-transitional stdext nbd tapctl libvhd oclock cdrom netdev xenopsd
Build XAPI
There is work going on to improve the build and moving it towards OPAM. But XAPI is still not packaged using OPAM. You can find this work here: https://github.com/jonludlam/xen-api/tree/ea-1254
A tree with additional fixes for building on CentOS 6.3 can be found here: https://github.com/JohnGarbutt/xen-api/tree/centos63
Now lets try and build this:
git clone https://github.com/JohnGarbutt/xen-api.git cd xen-api git checkout centos63 make set DEST_DIR=/tmp/xen-api mkdir $DEST_DIR make install
if you need to start again, try:
find -name "*.omc" | xargs rm -f make clean
Configuration
Now we need to configure things correctly. Append the following settings at the end of the appropriate file.
./.opam/system/etc/xenopsd.conf
sockets-path=/var/lib/xcp # use the legacy path for old xapi sockets-group=root hvmloader="/usr/lib/xen/boot/hvmloader" pygrub=/usr/lib64/xen/bin/pygrub vncterm=/usr/lib64/xen/bin/vncterm
/etc/xensource/network.conf
openvswitch
Building and Running OVS
While not needed, this is handy, and assumed below.
The info below is a modified version of these docs:
Some build prerequs:
yum install openssl-devel
autoconf 2.64 is required by 2.63 is packaged, so we try and get a more recent version from here:
- http://ftp.gnu.org/gnu/autoconf/autoconf-latest.tar.gz
- http://ftp.gnu.org/gnu/automake/automake-1.13.tar.xz
Due to issues with pkgconfig rpm, you then need to (see http://openvswitch.org/pipermail/discuss/2012-August/008064.html):
cp /usr/share/aclocal/pkg.m4 /usr/local/share/aclocal-1.13/
Now you can pull the latest ovs code and start to build it:
wget http://openvswitch.org/releases/openvswitch-1.4.5.tar.gz tar -xf openvswitch-1.4.5.tar.gz cd openvswitch-1.4.5 ./boot.sh ./configure make
Edit the Makefile so that you have the following setting, to match those expected by XenAPI:
prefix=/usr
Now we can install the binaries:
make install
Running OVS
First we need to turn off the bridge module, and check it is not loaded:
rmmod bridge lsmod | grep bridge
We can make it stop loading in future by:
echo >> /etc/modprobe.d/blacklist.conf <<EOF # Stop using bridge, using openvswitch instead blacklist bridge EOF
Now we need to load the kernel module for ovs:
insmod /lib/modules/3.4.32-6.el6.centos.alt.x86_64/kernel/net/openvswitch/openvswitch.ko
First time, we create the ovs db:
mkdir -p /usr/etc/openvswitch ovsdb-tool create /usr/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema
Now we can start the ovs config server (at the moment, this is without ssl):
ovsdb-server --remote=punix:/usr/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,manager_options \
--pidfile --detach
First time, you need to init the db:
ovs-vsctl --no-wait init
Now start main ovs deamon listening to same port:
ovs-vswitchd --pidfile --detach
Running xapi
There are a few more runtime dependencies:
yum install dmidecode
We can now run all the services we need:
./.opam/system/bin/xcp-fe # used to fork process service ./.opam/system/bin/xenopsd -config ./.opam/system/etc/xenopsd.conf # the component that talks to xen /opt/xensource/libexec/v6d #the license server /opt/xensource/libexec/xcp-networkd
Now we can run xapi:
/opt/xensource/bin/xapi # the api server, and database, and others things too
Running XAPI and OVS after restart
After you restart, by default, non of the services will restart. To help development, we can run theses services in a screen by doing:
git clone https://github.com/JohnGarbutt/xcp-devinstall.git cd xcp-devinstall ./run.sh screen -x xcp
Configure XenAPI Networking
No we have xapi running, we can ask it to detect all the physical interfaces:
host_uuid=`xe host-list --minimal` xe pif-scan host-uuid=$host_uuid # assuming a single pif pif_uuid=`xe pif-list --minimal` xe pif-reconfigure-ip uuid=$pif_uuid mode=dhcp xe host-management-reconfigure pif-uuid=$pif_uuid
To update OVS config we can do the following (TODO - find plugin, configure inventory):
host_uuid=`xe host-list --minimal` xe host-call-plugin host-uuid="$host_uuid" plugin="openvswitch-cfg-update" fn="update"
Configure XenAPI Storage
We first need to instal the storage SRs:
cd ~ git clone https://github.com/JohnGarbutt/xcp-storage-managers.git cd xcp-storage-managers yum install swig python-devel export DESTDIR=/ export PYTHON_INCLUDE=/usr/include/python2.6/ make
We need an SR to store the VM disks. It is best to add a new disk or partion, then just use that:
sr_uuid=`xe sr-create type=file device-config:location=/root/vhds name-label=localstorage sm-config:type=vhd` pool_uuid=`xe pool-list --minimal` xe pool-param-set uuid=$pool_uuid default-SR=$sr_uuid
Xen Cloud Platform Building Instructions (CentOS 5.6)
Building the xapi toolstack
The recommended way to build the xapi toolstack is in an environment similar to that in which it will be installed - therefore you'll need a CentOS 5.5 32-bit OS installed - we tend to use VMs for this.
For convenience, there are YUM repositories hosted on downloads.xen.org that contain the build dependencies of xapi. To enable these, create a a file /etc/yum.repos.d/xcp.repo with the following contents:
[xcp] name = xcp baseurl = http://downloads.xen.org/XCP/xcp-1.0-repo gpgcheck = 0
If you're building XCP unstable, replace '1.0' in the URL with 'unstable'.
Then execute the following commands:
rpm -e libaio yum install gcc autoconf automake tetex ghostscript java-1.6.0-openjdk java-1.6.0-openjdk-devel ant pam-devel python-devel zlib-devel openssl-devel dev86 rpm-build texinfo flex bison sharutils elfutils-devel ncurses-devel e4fsprogs-devel ocaml ocaml-findlib ocaml-getopt ocaml-type-conv ocaml-xmlm omake ocaml-xmlm-devel xen-devel xapi-libs-devel xapi-libs-utils
You'll also need mercurial or git depending upon whether you're building the unstable version of XCP or 1.0. These are both available in the EPEL repository (http://download.fedora.redhat.com/pub/epel/5/i386/repoview/)
Check out the xapi source:
hg clone http://xenbits.xen.org/XCP/xen-api.hg
or
git clone git@github.com:xen-org/xen-api.git
and then build it:
cd xen-api.hg make
or
cd xen-api make
It is also possible to build xapi in a non-CentOS 5.5 32-bit environment with the help of mock. Instructions for this are coming.
