Xen on 4 app servers

Needs Formatting Quite a lot of indentation issues

Objective

To have four application servers which will be running on top of Xen. These servers will have Debian as their operating system. These four servers will be hosting the applications which are critical to the functioning of organization. We are going to demonstrate how this is possible using Xen and Debian Linux distribution and Apache2.

System Description: Dell PowerEdge R710,Debian (amd64), 500GB Hard disk, 8GB RAM, Quad Core Processor.

Key Terms: Virtualization, Reverse Proxy on Apache2

Who should read this document:

• People/Organizations who want to have different servers for each application which will be dedicated to those applications without buying a separate hardware for these servers.
• System administrators who are new to Linux on Xen.
• Any one who is interested in testing virtualization.
• Curious people

Before you go to read this document familiarize yourself with DNS, Gateway and some basic networking stuff. Those who want to go a level higher they should learn how to use iptables on Linux.

What is virtualization?

• Virtualization is the name given to a concept where in you run multiple operating systems, at the same time, on one computer.
• Rather than running operating systems on the hardware, a small layer of software, called a hypervisor, runs between the operating system and hardware. The hypervisor's role is to provide a platform that can run multiple operating systems concurrently.
• These guest operating systems can be Windows, Linux, Solaris or any other operating system.

Test Infrastructure and Network

• We are going to install Debian on a Dell PowerEdge R710 Server. We have a network whose configuration is as follows

Dom0  with IP      192.168.0.100
DomU1 with IP      192.168.0.11
DomU2 with IP      192.168.0.12
DomU3 with IP      192.168.0.13
DomU4 with IP      192.168.0.14

All of these have a subnet mask of 255.255.255.0 and 192.168.0.1 is the gateway

Section A: Preparing the base system Dom0

Follow instructions at http://www.debian.org/releases/stable/installmanual to install the latest Debian version.

Step B Installing Xen on Debian

There are two ways to install Xen.

• Compile Xen from sources. See [[1]]
• Install a precompiled binary package.

We have chosen the second approach.

Installing Xen

apt-get install xen-linux-system
dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub

Setup networking. See Host_Configuration/Networking.

Creating LVM based DomU Images on Xen for Debian

Before we go on to proceed with LVM based images we will check for the LVM setup.You can skip this section if you have already set a LVM based setup and go to Creation of LVM based images.

• Check your hard disk structure

            dom0#fdisk      -l
        Device Boot      Start         End      Blocks              Id      System
        /dev/sda1               1        1216     9767488+          83      Linux
        /dev/sda2            1217       60801   478616512+           5        Extended
        /dev/sda5            1217        4863    29294496           82      Linux swap / Solaris
        /dev/sda6            4864       17021    97659103+          83      Linux
        /dev/sda7           17022       29179    97659103+          83      Linux
        /dev/sda8           29180       41337    97659103+          83      Linux
        /dev/sda9           41338       60801   156344548+          83      Linux

• First you need to install LVM:

 apt-get install lvm2

• Add each disk partition as a physical volume, into LVM.

                dom0#pvcreate /dev/sda6 /dev/sda7 /dev/sda8 /dev/sda9
                Physical volume "/dev/sda6" successfully created
                Physical volume "/dev/sda7" successfully created
                Physical volume "/dev/sda8" successfully created
                Physical volume "/dev/sda9" successfully created

You can run pvdisplay to learn about the current state of your physical volumes.

                  dom0:~# pvdisplay
                                --- Physical volume ---
                  PV Name               /dev/sda6
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               128
                  Allocated PE          23714
                  PV UUID               l8lIGR-246n-js5p-bb6y-w2zI-XCdy-r6H2CY
                                  --- Physical volume ---
                  PV Name               /dev/sda7
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               0
                  Allocated PE          23842
                  PV UUID               m6eqKv-ud3V-TJP2-p9E3-k4FK-3SxM-ZSzKrz
                                  --- Physical volume ---
                  PV Name               /dev/sda8
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               0
                  Allocated PE          23842
                  PV UUID               DGGrt0-Bxhn-JmYc-lq2O-acw8-DAt3-KaPC7P
                                  --- Physical volume ---
                  PV Name               /dev/sda9
                  VG Name
                  PV Size               149.10 GB / not usable 228.50 KB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              38170
                  Free PE               0
                  Allocated PE          38170
                  PV UUID               6ushwj-9Rse-cGGu-RXDk-2vTe-uSbw-6N5R42

• Now we will create our volume group virtualization and add /dev/sda6 - /dev/sda7- /dev/sda8 /dev/sda9

    dom0:~# vgcreate virtualization /dev/sda6 /dev/sda7 /dev/sda8 /dev/sda9
          Volume group "virtualization" successfully created
          dom0:~# vgscan
          Reading all physical volumes.  This may take a while...
          Found volume group "virtualization" using metadata type lvm2

Follow the instructions for installing a Debian guest, [[2]]

To use the newly created guest, execute

xl create -c /etc/xen/debian.cfg

To shut down the guest, execute 

xl shutdown debian

in dom0.

For more commands you can use xl help. For example, if you have four different DomU's installed, this will show:

     dom0:~# xl list
                Name         ID   Mem VCPUs      State   Time(s)
                Domain-0      0   875     8     r-----    361.4
                domu1         7  3072     4     -b----     32.9
                domu2         9  1024     4     -b----     32.0
                domu3         8  1024     4     -b----     34.0
                domu4         6  2048     4     -b----     32.2

If you want domu1 to start automatically at the next boot of the system, then do this: ln -s /etc/xen/domu1.cfg /etc/xen/auto

Configuration files

A sample configuration file for domU looks like this

kernel      = '/boot/vmlinuz-3.5.0'
ramdisk     = '/boot/initrd.img-3.5.0'
memory      = '2048'

#
#  Disk device(s).
#
disk        = [
                  'phy:/dev/virtualization/domu1-swap,xvda1,w',
                  'phy:/dev/virtualization/domu1-disk,xvda2,w',
              ]
#
#  Hostname
#
name        = 'domu1'
#
#  Networking
#
vif         = [ 'ip=192.168.0.11,mac=00:16:3E:66:03:E1' ]
#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'
vcpus = '4'

Setting up Networking

We are now going to set up each of our guests with networking. Each will have its own IP address. The result of this setup will look like:

          IP of Domu1:192.168.0.11                    Lets call it as a
          IP of Domu2:192.168.0.12                    Lets call it as b
          IP of Domu3:192.168.0.13                    Lets Call it as c
          IP of Domu4:192.168.0.14                    Lets Call it as d
                Ip of Dom0 :192.168.0.100  (Will behave as Gateway for DomU's)
                Lets call Dom0 as A
         Gateway for the network is 192.168.0.1
                        We will call it as G.

We have to enable IP forwarding for the hosts to appear on network:

dom0:~# echo 1 > /proc/sys/net/ipv4/ip_forward

In /etc/sysctl.conf uncomment net.ipv4.ip_forward=1

Now your hosts should appear on network. Inspite of the fact that that a,b,c,d are DomU's they are available on network as actual machines.

DNS should point to actual DNS of your network since with IP Forwarding enabled the DomU's will be able to access the proxy and rest of the network.

Now since you have enabled IP Forwarding so you should be able to ping from any other machine to you DomU's to make sure the things are working as expected you ping from C to a you should get a reply.

        ping   192.168.0.11 
        PING 192.168.0.11 (192.168.0.11) 56(84) bytes of data.
        64 bytes from 192.168.0.11: icmp_seq=1 ttl=64 time=3.32 ms
        64 bytes from 192.168.0.11: icmp_seq=2 ttl=64 time=0.045 ms
        ^C
        --- 192.168.0.11 ping statistics ---
        2 packets transmitted, 2 received, 0% packet loss, time 1008ms
        rtt min/avg/max/mdev = 0.045/1.685/3.326/1.641 ms

So DomU1 is accessible from C and in turn from the rest of the network. If you experience problems, ensure you have enabled ip forwarding in Dom0.

Xen has a default network managment which will create a bridge whose name will be same as your interface name. Follow instructions at Network_Configuration_Examples_(Xen_4.1+).

Install applications =

Now follow instructions to install and configure your desired applications.

Check points and possible errors

• Before you choose to install separate OS as DomU's calculte the amount of RAM they consume and swap space that to allocate them. When Dom's run they require a certain amount of memory to be allocated to them. Suppose you have 8 GB RAM and you decide to give 2GB Ram to each of the virtual hosts, you create then you won't be left with memory for Dom0. We planned to give 875 Mb memory to the Dom0 so we were left with 8192-875=7317Mb of Ram which we planned to give to the DomU's. The memory assigned to each domain is shown by the command xl list.

     dom0:~# xl list
                Name         ID   Mem VCPUs      State   Time(s)
                Domain-0      0   875     8     r-----    361.4
                domu1         7  3072     4     -b----     32.9
                domu2         9  1024     4     -b----     32.0
                domu3         8  1024     4     -b----     34.0
                domu4         6  2048     4     -b----     32.2

• Where domu1,domu2,domu3,domu4 are different dom's running.

If you encounter any problem doing above or want to give some suggestion that can be added in this page

• drop me a mail Tapas: mightydreams@gmail.com