Difference between revisions of "Xen via libvirt for OpenStack juno"

From Xen
Line 1: Line 1:
 
This document describes steps I take to setup a compute node with Xen via libvirt based on Ubuntu14.04 for Openstack juno. Openstack does not support this approach well as Xen via libvirt is in the group C of the hypervisor matrix support for Openstack. You can hardly find any tutorial online describing this approach and this might be the first. :-) Let's get started!
 
This document describes steps I take to setup a compute node with Xen via libvirt based on Ubuntu14.04 for Openstack juno. Openstack does not support this approach well as Xen via libvirt is in the group C of the hypervisor matrix support for Openstack. You can hardly find any tutorial online describing this approach and this might be the first. :-) Let's get started!
  +
----
 
 
'''Prerequisite''': Follow "OpenStack Installation Guide for Ubuntu 14.04" to setup the control node and network node. This involves lots of configurations and could take a day or two. Check that the control node and network node is working.
 
'''Prerequisite''': Follow "OpenStack Installation Guide for Ubuntu 14.04" to setup the control node and network node. This involves lots of configurations and could take a day or two. Check that the control node and network node is working.
   
  +
1. Add OpenStack juno to the repository.
Setup a Xen via libvirt compute node for Ubuntu14.04.
 
 
# Add OpenStack juno to the repository.
 
 
apt-get update
 
apt-get update
 
apt-get install software-properties-common
 
apt-get install software-properties-common
 
add-apt-repository cloud-archive:juno
 
add-apt-repository cloud-archive:juno
 
apt-get update
 
apt-get update
  +
2. Install nova-compute-xen and sysfsutils
  +
apt-get install nova-compute-xen sysfsutils python-novaclient
  +
3. Edit the /etc/nova/nova.conf and add all the following lines. You can also follow steps in the installation guide to configure nova in the compute node.
  +
** In the [default] section, add the following. <br/>
  +
[default]
  +
...
  +
rpc_backend = rabbit
  +
rabbit_host = controller
  +
rabbit_password = RABBIT_PASS
  +
auth_strategy = keystone
  +
my_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
  +
vnc_enabled = True
  +
vncserver_listen = 0.0.0.0
  +
vncserver_proxyclient_address = MANAGEMENT_INTERFACE_IP_ADDRESS
  +
novncproxy_base_url = http://controller:6080/vnc_auto.html
  +
verbose = True
  +
  +
MANAGEMENT_INTERFACE_IP_ADDRESS is the IP address of the management network interface, typically 10.0.0.31 for the first compute node.
   
  +
** Add keystone_authtoken section <br />
 
  +
[keystone_authtoken]
2). Install nova-compute-xen and sysfsutils
 
  +
auth_uri = http://controller:5000/v2.0
# apt-get install nova-compute-xen sysfsutils python-novaclient
 
  +
identity_uri = http://controller:35357
3). Edit the /etc/nova/nova.conf and add all the following lines. You can also follow steps in the installation guide to configure nova in the compute node.
 
  +
admin_tenant_name = service
3.1) In the [default] section, add the following.
 
  +
admin_user = nova
[default]...rpc_backend = rabbit
 
  +
admin_password = NOVA_PASS
rabbit_host = controller
 
  +
** Add glance section
rabbit_password = RABBIT_PASS
 
  +
[glance]
auth_strategy = keystone
 
  +
host=controller
my_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
 
vnc_enabled = True
 
vncserver_listen = 0.0.0.0
 
vncserver_proxyclient_address = MANAGEMENT_INTERFACE_IP_ADDRESS
 
novncproxy_base_url = http://controller:6080/vnc_auto.html
 
verbose = True
 
MANAGEMENT_INTERFACE_IP_ADDRESS is the IP address of the management network interface, typically 10.0.0.31 for the first compute node.
 
 
3.2) Add keystone_authtoken section
 
[keystone_authtoken]
 
auth_uri = http://controller:5000/v2.0
 
identity_uri = http://controller:35357
 
admin_tenant_name = service
 
admin_user = nova
 
admin_password = NOVA_PASS
 
3.3) Add glance section
 
[glance]
 
host=controller
 
 
4. Verify /etc/nova/nova-compute.conf content is as following.
 
4. Verify /etc/nova/nova-compute.conf content is as following.
[DEFAULT]
+
[DEFAULT]
compute_driver=libvirt.LibvirtDriver
+
compute_driver=libvirt.LibvirtDriver
[libvirt]
+
[libvirt]
virt_type=xen
+
virt_type=xen
 
5. Installing nova-compute-xen doesn't install xencommons script for us. However, we need xencommons to start the qemu process to provide block device backend service for qemu disks. I compile and install xen-4.4 from source, to get the xencommons script. Note this is likely not the preferred approach, and please update it if you know how.
 
5. Installing nova-compute-xen doesn't install xencommons script for us. However, we need xencommons to start the qemu process to provide block device backend service for qemu disks. I compile and install xen-4.4 from source, to get the xencommons script. Note this is likely not the preferred approach, and please update it if you know how.
   
5.1 Download xen-4.4 source code, compile and install.
+
** Download xen-4.4 source code, compile and install.
  +
apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 \
# apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils libyajl-dev gettext libpixman-1-dev
 
  +
module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended \
# apt-get build-dep xen
 
  +
texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial make gcc libc6-dev \
# cd xen;
 
  +
zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev \
# ./configure
 
  +
iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils \
# make dist -j16
 
  +
libyajl-dev gettext libpixman-1-dev
# make install
 
  +
apt-get build-dep xen
  +
cd xen;
  +
./configure
  +
make dist -j16
  +
make install
   
5.2 Enable startup services
+
** Enable startup services
# update-rc.d xencommons defaults
+
update-rc.d xencommons defaults
# update-rc.d xen-watchdog defaults
+
update-rc.d xen-watchdog defaults
   
5.3 Install virt-manager
+
** Install virt-manager
# apt-get install virt-manager
+
apt-get install virt-manager
   
 
If we do not install virt-manager, when you run "xl", it may say "xl: error while loading shared libraries: libxlutil.so.4.3: cannot open shared object file: No such file or directory"
 
If we do not install virt-manager, when you run "xl", it may say "xl: error while loading shared libraries: libxlutil.so.4.3: cannot open shared object file: No such file or directory"
   
 
6. Install qemu-2.0.2 with a patch fixing unmapping of persistent grants. Current qemu releases (including 2.0.2, 2.1.2 and 2.2.0-rc1) do not have this patch included and this will result in dom0 kernel crashes when creating a Xen domU from OpenStack GUI (horizon).
 
6. Install qemu-2.0.2 with a patch fixing unmapping of persistent grants. Current qemu releases (including 2.0.2, 2.1.2 and 2.2.0-rc1) do not have this patch included and this will result in dom0 kernel crashes when creating a Xen domU from OpenStack GUI (horizon).
# wget https://github.com/xinglin/qemu-2.0.2/archive/master.zip
+
wget https://github.com/xinglin/qemu-2.0.2/archive/master.zip
# unzip master.zip
+
unzip master.zip
# cd qemu-2.0.2-master/
+
cd qemu-2.0.2-master/
# ./configure
+
./configure
# make -j16
+
make -j16
# make install
+
make install
   
6.1 Update /etc/init.c/xencommons to use the new version of qemu, rather than the default one.
+
** Update /etc/init.c/xencommons to use the new version of qemu, rather than the default one.
  +
<code>
--- /etc/init.d/xencommons.bk 2014-11-14 10:57:32.183529318 -0700
 
+++ /etc/init.d/xencommons 2014-11-14 10:56:42.474535348 -0700
+
--- /etc/init.d/xencommons.bk 2014-11-14 10:57:32.183529318 -0700
  +
+++ /etc/init.d/xencommons 2014-11-14 10:56:42.474535348 -0700
@@ -118,6 +118,7 @@ do_start () {
 
  +
@@ -118,6 +118,7 @@ do_start () {
 
${SBINDIR}/xenconsoled --pid-file=$XENCONSOLED_PIDFILE $XENCONSOLED_ARGS
 
${SBINDIR}/xenconsoled --pid-file=$XENCONSOLED_PIDFILE $XENCONSOLED_ARGS
 
echo Starting QEMU as disk backend for dom0
 
echo Starting QEMU as disk backend for dom0
 
test -z "$QEMU_XEN" && QEMU_XEN="${LIBEXEC}/qemu-system-i386"
 
test -z "$QEMU_XEN" && QEMU_XEN="${LIBEXEC}/qemu-system-i386"
+ QEMU_XEN="/usr/local/bin/qemu-system-i386"
+
+ QEMU_XEN="/usr/local/bin/qemu-system-i386"
 
$QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
 
$QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
 
-monitor /dev/null -serial /dev/null -parallel /dev/null \
 
-monitor /dev/null -serial /dev/null -parallel /dev/null \
 
-pidfile $QEMU_PIDFILE
 
-pidfile $QEMU_PIDFILE
  +
</code>
   
 
7. Create a link at /usr/bin for pygrub
 
7. Create a link at /usr/bin for pygrub
Line 86: Line 92:
   
 
8. Install and configure network component in compute node.
 
8. Install and configure network component in compute node.
Follow steps in this guide. Note, in /etc/neutron/neutron.conf, I did not set "allow_overlapping_ips = True" in the default section, because it is said to set this property to False if Neutron and the nova security groups are used together.
+
Follow steps in this [http://docs.openstack.org/juno/install-guide/install/apt/content/neutron-compute-node.html guide]. Note, in /etc/neutron/neutron.conf, I did not set "allow_overlapping_ips = True" in the default section, because it is said to set this property to False if Neutron and the nova security groups are used together.
 
http://docs.openstack.org/juno/install-guide/install/apt/content/neutron-compute-node.html
 
http://docs.openstack.org/juno/install-guide/install/apt/content/neutron-compute-node.html
   
8. Reboot the machine.
+
9. Reboot the machine.

Revision as of 22:52, 14 November 2014

This document describes steps I take to setup a compute node with Xen via libvirt based on Ubuntu14.04 for Openstack juno. Openstack does not support this approach well as Xen via libvirt is in the group C of the hypervisor matrix support for Openstack. You can hardly find any tutorial online describing this approach and this might be the first. :-) Let's get started!


Prerequisite: Follow "OpenStack Installation Guide for Ubuntu 14.04" to setup the control node and network node. This involves lots of configurations and could take a day or two. Check that the control node and network node is working.

1. Add OpenStack juno to the repository.

 apt-get update
 apt-get install software-properties-common
 add-apt-repository cloud-archive:juno
 apt-get update

2. Install nova-compute-xen and sysfsutils

 apt-get install nova-compute-xen sysfsutils python-novaclient

3. Edit the /etc/nova/nova.conf and add all the following lines. You can also follow steps in the installation guide to configure nova in the compute node.

    • In the [default] section, add the following.
 [default]
 ...
 rpc_backend = rabbit
 rabbit_host = controller
 rabbit_password = RABBIT_PASS
 auth_strategy = keystone
 my_ip = MANAGEMENT_INTERFACE_IP_ADDRESS
 vnc_enabled = True
 vncserver_listen = 0.0.0.0
 vncserver_proxyclient_address = MANAGEMENT_INTERFACE_IP_ADDRESS
 novncproxy_base_url = http://controller:6080/vnc_auto.html
 verbose = True
 
 MANAGEMENT_INTERFACE_IP_ADDRESS is the IP address of the management network interface, typically 10.0.0.31 for the first compute node.
    • Add keystone_authtoken section
 [keystone_authtoken]
 auth_uri = http://controller:5000/v2.0
 identity_uri = http://controller:35357
 admin_tenant_name = service
 admin_user = nova
 admin_password = NOVA_PASS
    • Add glance section
 [glance]
 host=controller

4. Verify /etc/nova/nova-compute.conf content is as following.

 [DEFAULT]
 compute_driver=libvirt.LibvirtDriver
 [libvirt]
 virt_type=xen

5. Installing nova-compute-xen doesn't install xencommons script for us. However, we need xencommons to start the qemu process to provide block device backend service for qemu disks. I compile and install xen-4.4 from source, to get the xencommons script. Note this is likely not the preferred approach, and please update it if you know how.

    • Download xen-4.4 source code, compile and install.
    apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 \
                           module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended \
                           texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial make gcc libc6-dev \
                           zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev \
                           iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml ocaml-findlib libx11-dev bison flex xz-utils \
                           libyajl-dev gettext libpixman-1-dev
    apt-get build-dep xen
    cd xen; 
    ./configure
    make dist -j16
    make install 
    • Enable startup services
   update-rc.d xencommons defaults 
   update-rc.d xen-watchdog defaults
    • Install virt-manager
  apt-get install virt-manager

If we do not install virt-manager, when you run "xl", it may say "xl: error while loading shared libraries: libxlutil.so.4.3: cannot open shared object file: No such file or directory"

6. Install qemu-2.0.2 with a patch fixing unmapping of persistent grants. Current qemu releases (including 2.0.2, 2.1.2 and 2.2.0-rc1) do not have this patch included and this will result in dom0 kernel crashes when creating a Xen domU from OpenStack GUI (horizon).

    wget https://github.com/xinglin/qemu-2.0.2/archive/master.zip
    unzip master.zip
    cd qemu-2.0.2-master/
     ./configure
     make -j16
     make install
    • Update /etc/init.c/xencommons to use the new version of qemu, rather than the default one.

       --- /etc/init.d/xencommons.bk	2014-11-14 10:57:32.183529318 -0700
       +++ /etc/init.d/xencommons	2014-11-14 10:56:42.474535348 -0700
       @@ -118,6 +118,7 @@ do_start () {
	${SBINDIR}/xenconsoled --pid-file=$XENCONSOLED_PIDFILE $XENCONSOLED_ARGS
	echo Starting QEMU as disk backend for dom0
	test -z "$QEMU_XEN" && QEMU_XEN="${LIBEXEC}/qemu-system-i386"
   +	QEMU_XEN="/usr/local/bin/qemu-system-i386"
	$QEMU_XEN -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize \
		-monitor /dev/null -serial /dev/null -parallel /dev/null \
		-pidfile $QEMU_PIDFILE

7. Create a link at /usr/bin for pygrub

    # ln -s /usr/local/bin/pygrub /usr/bin/pygrub

8. Install and configure network component in compute node. Follow steps in this guide. Note, in /etc/neutron/neutron.conf, I did not set "allow_overlapping_ips = True" in the default section, because it is said to set this property to False if Neutron and the nova security groups are used together. http://docs.openstack.org/juno/install-guide/install/apt/content/neutron-compute-node.html

9. Reboot the machine.