Xen on 4 app servers

Needs Formatting Quite a lot of indentation issues

Virtualization with Xen on Debian Lenny

Objective

To have four application servers which will be running on top of a Xen Server. These servers will have Debian Lenny as their operating system. These four servers will be hosting the applications which are critical to the functioning of organization. We are going to demonstrate how this is possible using Xen and Debian Linux distribution and Apache2.

System Description: Dell PowerEdge R710,Debian Lenny (amd64) bit, 500GB Hard disk, 8GB RAM, Quad Core Processor.

Key Terms: Virtualization, Reverse Proxy on Apache2

Who should read this document:

• People/Organizations who want to have different servers for each application which will be dedicated to those applications without buying a separate hardware for these servers.System administrators who are new to Xen on Linux.Any one who is interested to test virtualization for their applications.Some one who wants to understand how things work for their knowledge. Before you go to read this document familiarize yourself with DNS, Gateway and some basic networking stuff. That will be fine. Those who want to go a level higher they should learn use of IPTABLES on Linux. You do not need to worry about IPABLES if you are not clear with that part.

What is virtualization

• Virtualization is the name given to a concept where in you run different many different Operating Systems as Guest.
• Operating systems using a layer which is known as hypervisor. In this document we have used Xen.These Guest Operating Systems can be Windows,Linux,Solaris or any other operating sytems of your choice and you can always have a different operating system as Host OS. There are commercial applications such as vmware which can be used in place of Xen. KVM,OpenVZ are among the other applications which can be used in place of Xen. Which hypervisor you want to use is totally upon your choice.Discussion of Virtualization is beyond the scope of this document and this small page about Virtualization should give you a basic insight about Virtualization. If you want to read more about Virtualization you should search internet. You can read books about Virtualization here are a few to name.
  • Amazon Link
• http://www.amazon.com/Running-Xen-Hands-Guide-Virtualization/dp/0132349663
• Running Xen: A Hands-On Guide to the Art of Virtualization by Jeanna N. Matthews; Eli M. Dow; Todd Deshane; Wenjin Hu; Jeremy Bongio; Patrick F. Wilbur; Brendan
  • Johnsonby Jeanna N. Matthews; Eli M. Dow; Todd Deshane; Wenjin Hu; Jeremy Bongio; Patrick F. Wilbur;Brendan Johnson
  • Another is : Xen Virtualization
• http://www.amazon.com/Xen-Virtualization-practical-supporting-hypervisor/dp/1847192483 By:PrabhakarChaganti
• If you are already aware of basics of Virtualization then you may skip this section and move ahead to next section.

Test Infrastructure and Network

• We are going to install Debian on Dell PowerEdge R710 Server. We have a network whose configuration is as follows

Dom0  with IP      192.168.0.100 (This will be our Xen Server)
DomU1 with IP      192.168.0.11
DomU2 with IP      192.168.0.12
DomU3 with IP      192.168.0.13
DomU4 with IP      192.168.0.14

• Subnet for all of the above is 255.255.255.0 . Gateway to network is 192.168.0.1

Section A: Preparing the base system Dom0

• 1 Get a copy of Debian amd64 version. This can be obtained from here
• Download Debian here
• 2 A firmware upgrade was needed at the time of writing this guide to make LAN card work.
• Firmware update which made R710 lan card work

   [[1]]

• [[2]]
• 3) Install Debian Lenny (amd64) on Dell Poweredge R710.
• Choose 10GB partition to install.We choose 10GB to install base system on it. Choose 30GB for swap space.These specifications are not strictly necessary you can have your own choices depending on your use. Rest of the 460GB leave empty creating 4 different partitions. Size of partitions remaining is 160,100,100,100 GB partitions. You can leave them as it is or can create a LVM group which consist of all the above partition in same volume group.After the base system Debian Lenny has been installed and you have upgraded firmware and booted the base system,you should update sources.list file.In case you run through some errors in your netwok or LAN before proceeding to the next section go through the errors section.
• Do following

  xenserver#cat >> /etc/apt/sources.list
  deb http://ftp.us.debian.org/debian/ lenny main
  deb-src http://ftp.us.debian.org/debian/ lenny main
  deb http://security.debian.org/ lenny/updates main contrib
  deb-src http://security.debian.org/ lenny/updates main contrib
  deb http://ftp.us.debian.org/debian/ lenny non-free
  deb http://ftp.us.debian.org/debian/ lenny contrib

• and press enter.Your /etc/apt/sources.list should look like this

   deb http://ftp.us.debian.org/debian/ lenny main
   deb-src http://ftp.us.debian.org/debian/ lenny main
   deb http://security.debian.org/ lenny/updates main contrib
   deb-src http://security.debian.org/ lenny/updates main contrib
   deb http://ftp.us.debian.org/debian/ lenny non-free
   deb http://ftp.us.debian.org/debian/ lenny contrib

• Note: After you have installed DomU's the above mirrors can also be used onDomU's
• 4) Update your network settings i.e. IP, gateway,DNS,netmask etc.
• vi /etc/network/interfaces

# The loopback network interface auto lo iface lo inet loopback
# The primary network interface
  #allow-hotplug eth0
#iface eth0 inet dhcp
auto eth0 iface eth0
inet static address 192.168.0.100
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
gateway 192.168.0.1

• Then restart your network:

/etc/init.d/networking restart

• Then edit /etc/hosts.

 vi /etc/hosts

  127.0.0.1       localhost.localdomain           localhost
  192.168.0.100  xenserver.lan.ernet.in           xenserver

• Modify the settings for your network replace the domain names with what ever is applicable for you.
• Edit /etc/resolv.conf and add some nameservers:

vi /etc/resolv.conf

nameserver 192.168.1.24
nameserver 192.168.1.25

where 192.168.1.24 and 192.168.1.25 are DNS servers.

They should contain entries which are relevant for you.

• 5) apt-get update
• 6)Do apt-get install ssh vim-full

Step B Installing Xen on Debian Lenny

• There are two ways to install Xen.
• a) Compile Xen from sources.This one you can read here
  • [[3]]
  • b) Install a precompiled binary for your architecture.
  • .

We have chosen the second approach.

Step 1) Installing Xen Before going down

while installing xen-hypervisor-3.2-1-amd64 two packages xen-utils-3.2-1 and xenstore-utils also gets installed so you need not to install it both of them along with packages mentioned below.

apt-get install xen-hypervisor-3.2-1-amd64  xen-linux-system-2.6.26-2-xen-amd64     xenwatch xen-shell  xen-tools

• To check what these packages contain you can check
• http://packages.debian.org/lenny/xen-hypervisor-3.2-1-amd64
• http://packages.debian.org/lenny/xen-linux-system-2.6.26-2-xen-amd64
• http://packages.debian.org/lenny/xen-utils-3.2-1
• http://packages.debian.org/lenny/xenstore-utils
• http://packages.debian.org/lenny/xenwatch
• http://packages.debian.org/lenny/xen-shell
• http://packages.debian.org/lenny/xen-tools
• While installing above note the messages some of them download the xen-utils package so you need not to re
• install them again. Before you proceed to check that which package is installed you can do so by dpkg -s <package name>

Step2) open /etc/xen/xend-config.sxp

• vi /etc/xen/xend-config.sxp
• We have used a bridged network setup if you want to use NAT or routed setup based on IPTABLES then there will be a

       different setup.

• Step 3) uncomment the line

  (network-script network-bridge)

• and comment out the line

(network-script network-dummy)

• Also make sure that the line (vif-script vif-bridge) is enabled:
• The above file should appear some thing like this

[...]
(network-script network-bridge)
[...]
#(network-script network-dummy)
 [...]
(vif-script vif-bridge)
 [...]

• Then reboot the
• system: reboot
• and check
• check uname -r and your new Xen kernel should show up:

xenserver:~# uname -r         2.6.26-2-xen-amd64

• you can do an xm info an output like some thing as follows should come up

  xenserver:~# xm info
    host                      :  xenserver
    release                   : 2.6.26-2-xen-amd64
    version                   :  #1 SMP Wed Mar 10 00:29:48 UTC 2010
    machine                   :  x86_64
    nr_cpus                   : 8
    nr_nodes                  : 1
    cores_per_socket          : 4
    threads_per_core          : 2
    cpu_mhz                   : 2261
    hw_caps                   :bfebfbff:28100800:00000000:00000140:009ce3bd:00000000:00000001
    total_memory              : 8182
    free_memory               : 6
    node_to_cpu               : node0:0-7
    xen_major                 : 3
    xen_minor                 : 2
    xen_extra                 : -1
    xen_caps                  : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32  hvm-3.0-x86_32p hvm-3.0-x86_64
    xen_scheduler             : credit
    xen_pagesize              : 4096
    platform_params           : virt_start=0xffff800000000000
    xen_changeset             : unavailable
    cc_compiler               : gcc version 4.3.1 (Debian 4.3.1-2)
    cc_compile_by             : waldi
    cc_compile_domain         : debian.org
    cc_compile_date           : Sat Jun 28 09:32:18 UTC 2008
    xend_config_format        : 4

• . = Step C: Creating LVM based DomU Images on Xen for Debian Lenny =
• Before we go on to proceed with LVM based images we will check for the LVM setup.You can skip this section if you have already set a LVM based setup and go to Creation ofLVM based images.
• Check your hard disk structure

            xenserver#fdisk      -l
        Device Boot      Start         End      Blocks              Id      System
        /dev/sda1               1        1216     9767488+          83      Linux
        /dev/sda2            1217       60801   478616512+           5        Extended
        /dev/sda5            1217        4863    29294496           82      Linux swap / Solaris
        /dev/sda6            4864       17021    97659103+          83      Linux
        /dev/sda7           17022       29179    97659103+          83      Linux
        /dev/sda8           29180       41337    97659103+          83      Linux
        /dev/sda9           41338       60801   156344548+          83      Linux

• First you need to update some packages.

 apt-get install lvm2

• The above system configuration has 4 hard disks partitions

                /dev/sda6
                /dev/sda7
                /dev/sda8
                /dev/sda9

                xenserver#pvcreate /dev/sda6 /dev/sda7 /dev/sda8 /dev/sda9
                Physical volume "/dev/sda6" successfully created
                Physical volume "/dev/sda7" successfully created
                Physical volume "/dev/sda8" successfully created
                Physical volume "/dev/sda9" successfully created

• Check point if you did not made 4 partitions while installing you need to do some home work.
• You can run pvdisplay to learn about the current state of your physical volumes.

                  xenserver:~# pvdisplay
                                --- Physical volume ---
                  PV Name               /dev/sda6
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               128
                  Allocated PE          23714
                  PV UUID               l8lIGR-246n-js5p-bb6y-w2zI-XCdy-r6H2CY
                                  --- Physical volume ---
                  PV Name               /dev/sda7
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               0
                  Allocated PE          23842
                  PV UUID               m6eqKv-ud3V-TJP2-p9E3-k4FK-3SxM-ZSzKrz
                                  --- Physical volume ---
                  PV Name               /dev/sda8
                  VG Name
                  PV Size               93.13 GB / not usable 2.22 MB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              23842
                  Free PE               0
                  Allocated PE          23842
                  PV UUID               DGGrt0-Bxhn-JmYc-lq2O-acw8-DAt3-KaPC7P
                                  --- Physical volume ---
                  PV Name               /dev/sda9
                  VG Name
                  PV Size               149.10 GB / not usable 228.50 KB
                  Allocatable           yes (but full)
                  PE Size (KByte)       4096
                  Total PE              38170
                  Free PE               0
                  Allocated PE          38170
                  PV UUID               6ushwj-9Rse-cGGu-RXDk-2vTe-uSbw-6N5R42

• Now we will create our volume group virtualization and add /dev/sda6 - /dev/sda7- /dev/sda8 /dev/sda9

    xenserver:~# vgcreate virtualization /dev/sda6 /dev/sda7 /dev/sda8 /dev/sda9
          Volume group "virtualization" successfully created
          xenserver:~# vgscan
          Reading all physical volumes.  This may take a while...
          Found volume group "virtualization" using metadata type lvm2

• You do not need to create LVM here. LVM will be created by the xen-create-image
  • it will create a swap and root partition which will be an LVM image.

Creation of LVM based images

    vi /etc/xen-tools/xen-tools.conf
                [........]
                 lvm = virtualization
                [.......]
                [...]
                dist   = lenny     # Default distribution to install.
                [...]
                gateway   = 192.168.0.100
                 #(your gateway will be same as your Dom0 for DomU's)
                netmask   = 255.255.255.0
                broadcast = 192.168.0.255
                [.........]
                size   = 150Gb      # Disk image size.
                memory = 2048Mb    # Memory size
                swap   = 4096Mb    # Swap size
                # noswap = 1      # Don't use swap at all for the new system.
                fs     = ext3     # use the EXT3 filesystem for the disk image.
                dist   = lenny    # Default distribution to install.
                image  = sparse   # Specify sparse vs. full disk images.
                        [........]
                # Uncomment the following line if you wish to interactively setup
                # a new root password for images.
                #
                 passwd = 1
                 [..........]
                mirror = http://ftp2.de.debian.org/debian/
                        [...........]
                serial_device = hvc0
                [...]
                disk_device = xvda #default

• Here mirror is pointing to a debian repository which is used for installation.If you use a loop mounted CDROM then you can replace it with the location of mounted directory. I have mentioned both the methods below you can choose any.

Do not forget to chage serial_device = hvc0 and disk_device =xvda as shown above.

• What ever you write above will be replaced by anything that you will specify on command line.
  • Most Imp: Specify IP of gateway as the IP of your Dom0

We do  not need to make or compile a DomU kernel.We can use default debian kernels on   the installation media.  Default  installation OS type above in file was etch where as we are using lenny so replace it by that. Size above 160GB can vary  depending upon availability of  hard disk space you have.

• The dir line specifies where the virtual machine images will be stored. '

dist specifies the distribution to be installed in the virtual machines (Debian Lenny) (there's a comment in the file

• that explains what distributions are currently supported).

The passwd = 1 line makes that you can specify a root password when you create a new    guest domain. Make sure that you  uncomment the lvm line and fill in the name of your     volume group (virtualization is the volumegroup here). At the same  time make sure that the dir line is commented out.

• It is very important that you add the line serial_device = hvc0 (not xvc0 or tty1)otherwise the console of your
• virtual machines might not work properly.

Method 1 (If you do not want to go for internet one you may skip it and go to Method2) Using internet based repository

        xenserver#xen-create-image --hostname=domu1 --size=160GB --swap=4096Mb --ip=192.168.0.11 --memory=2048 --arch=amd64 --role=udev --force --install-method=debootstrap

• What ever you specify on the command line here will over write the
• /etc/xen-tools/xen-tools.conf

You may specify a different internet mirror above. Note –role-udev above do not miss it. Method 2 (You can skip it and go to section image ) Using CD as your local repository

• http://cdimage.debian.org/debian-cd/5.0.4/amd64/iso-cd/
• If you have the CD image burnt on to a CD do the following insert Debian CD in CDROM

dd if=/dev/cdrom of=/home/xenserver/debian.iso bs=1024

• After a few minutes you will get a message saying that the CD has been copied.Same
• command will work for DVD also.You need to check your DVD rom in your /dev .
  • Now make a directory named local in your home directory i.e.(the name local is not strictly required you may choose
  • what ever you wish it)

        mkdir /home/xenserver/local
        mount   -o loop -t iso9660 /home/xenserver/debian.iso   /home/xenserver/local

• You can navigate to directory local and see the contents of CDROM in local directory.

       xen-create-image --hostname=domu1 --size=150GB --swap=4096Mb --ip=192.168.0.11 --memory=2048 --arch=amd64
           --role=udev --force   --install-method=debootstrap --install-source=/home/xenserver/local

• Image
  • The xen-create-image command will now create the domu1 virtual machine for us. This will take a few minutes. The output should be similar to this one:

xen-create-image --hostname=domu1 --size=150GB --swap=4096Mb --ip=192.168.0.11 --memory=2048 --arch=amd64 --role=udev --force  --install-method=debootstrap --install-source=/home/xenserver/local 

General Information
--------------------
Hostname       :  domu1
Distribution   :  lenny
Partitions     :  swap4096Mb (swap)
                  /150Gb   (ext3)
Image type     :  sparse
Memory size    :  2096Mb
Kernel path    :  /boot/vmlinuz-2.6.26-2-xen-amd64
Initrd path    :  /boot/initrd.img-2.6.26-2-xen-amd64
Networking Information
----------------------
IP Address 1   : 192.168.0.11
Netmask        : 255.255.255.0
Broadcast      : 192.168.0.255
Gateway        : 192.168.0.100
Creating partition image:
Done
Creating swap on
Done
Creating partition image:
Done
Creating ext3 filesystem on
Done
Installation method: debootstrap
Done
Running hooks
Done
Role: udev
        File: /etc/xen-tools/role.d/udev
Role script completed.
Creating Xen configuration file
Done
Setting up root password
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
All done
Logfile produced at:
         /var/log/xen-tools/domu1.log
xenserver:~#

• There should now be a domu1 configuration file - /etc/xen/domu1.cfg. If you have a dual-core or quad-core CPU and want the virtual machine to use all CPU cores, please add the line vcpus =
  • '2' or vcpus = '4' to the configuration file If you are wondering that we have not created Logical Volumes then just notice the outputs of above commands which have
  • created separate root and swap partitions.You can confirm it by command lvdisplay

             --- Logical volume ---
          LV Name                /dev/virtualization/domu1-swap
          VG Name                virtualization
          LV UUID                jN8OjS-kngH-v5Na-NYBJ-Oy3H-QQyL-M0z8zk
          LV Write Access        read/write
          LV Status              available
          # open                 1
          LV Size                4.00 GB
          Current LE             1024
          Segments               1
          Allocation             inherit
          Read ahead sectors     auto
          - currently set to     256
          Block device           254:0
                          --- Logical volume ---
          LV Name                /dev/virtualization/domu1-disk
          VG Name                virtualization
          LV UUID                LEx5WN-Dnkc-pjqJ-WyIJ-6W7n-jTHo-8dDPre
          LV Write Access        read/write
          LV Status              available
          # open                 1
          LV Size                150.00 GB
          Current LE             38400
          Segments               2
          Allocation             inherit
          Read ahead sectors     auto
          - currently set to     256
          Block device           254:1

• Now you have one DomU1 up same process you can apply to create other instances.

= Step D:Using the created Virtual Machines i.e. Installed  Guest Operating Systems =

• To be able to use the newly created guest i.e. Virtual machine whose name is domu
  • xm create /etc/xen/domu1.cfg

 and then run
xenserver#xm console domu1 The output should look like this:

        xenserver:~#xm list
        Name                                          ID  Mem   VCPUs     State  Time(s)
        Domain-0                                     0   8192    8                r----- 398.2
        domu1                                         1   2048    4                -b---- 2.8

To shut down domu1, do this: xm shutdown domu1

• For more commands you can use xm help.
• Now suppose you have 4 different DomU's installed

     xenserver:~# xm list
                Name         ID   Mem VCPUs      State   Time(s)
                Domain-0      0   875     8     r-----    361.4
                domu1         7  3072     4     -b----     32.9
                domu2         9  1024     4     -b----     32.0
                domu3         8  1024     4     -b----     34.0
                domu4         6  2048     4     -b----     32.2

If you want domu1 to start automatically at the next boot of the system, then do this: ln -s /etc/xen/domu1.cfg /etc/xen/auto

Step E: Knowing the configuration files

• A sample configuration file for domU looks like this

#
# Configuration file for the Xen instance vignettes, created
# by xen-tools 3.9 on Fri Apr  9 09:01:18 2010.
#
#
#  Kernel + memory size
#
kernel      = '/boot/vmlinuz-2.6.26-2-xen-amd64'
ramdisk     = '/boot/initrd.img-2.6.26-2-xen-amd64'
memory      = '2048'
#
#  Disk device(s).
#
root        = '/dev/xvda2 ro'
disk        = [
                  'phy:/dev/virtualization/domu1-swap,xvda1,w',
                  'phy:/dev/virtualization/domu1-disk,xvda2,w',
              ]
#
#  Hostname
#
name        = 'domu1'
#
#  Networking
#
vif         = [ 'ip=192.168.0.11,mac=00:16:3E:66:03:E1' ]
#
#  Behaviour
#
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'
vcpus = '4'
~
        If you have a dual-core or quad-core CPU and want the virtual machine to use all CPU    cores, please add the line vcpus = '2' or vcpus = '4' to the configuration file

• A list of virtual machines that were created with the xen-create-image command can be found by xen-list-images

Step F:Setting up of Networking on Xenserver and DomU's

• When you install 4 DomU's with Ips on them it is as good as bringing 4 hosts on network.
  • In our case
    • We have following

          IP of Domu1:192.168.0.11                    Lets call it as a
          IP of Domu2:192.168.0.12                    Lets call it as b
          IP of Domu3:192.168.0.13                    Lets Call it as c
          IP of Domu4:192.168.0.14                    Lets Call it as d
                Ip of Dom0 :192.168.0.100  (Will behave as Gateway for DomU's)
                Lets call Dom0 as A
         Gateway for the network is 192.168.0.1
                        We will call it as G.

• So our network structure is some thing like that
  • DomU1 DomU2 DomU3 DomU4

192.168.0.11    192.168.0.12        192.168.0.13          192.168.0.14

• a b c d
  • Dom0
  • 192.168.0.100 [[[XenServer]]]
  • IP Forwarding Enabled on Dom0
    • Figure1
• We have to enable Ip forwarding.For the hosts to appear on network.
  • xenserver:~# echo 1 > /proc/sys/net/ipv4/ip_forward
• also in /etc/sysctl.conf uncomment net.ipv4.ip_forward=1
• Now your hosts should be appearing on network.Do not have any other host have the same IP as DomU's otherwise IP

collision will occur.So suppose you have a machine C  the structure is now like this

      A(Xen Server)
                192.168.0.100
                      a
                192.168.0.11
                (DomU1)
                      b
                192.168.0.12
                (DomU2)
                     c
                192.168.0.13                                                     G (Gateway)
                (DomU3)                                                          192.168.0.1
                     d
                192.168.0.14
                  (DomU4)
                   C
                192.168.0.41
            Some machine on network

• Note : Inspite of the fact that that a,b,c,d are DomU's they are available on network as
  • actual machines. This is what you should have achieved by now.
    • Figure 2

 When you login to any of the DomU you should set the gateway as Dom0.

• DNS can be pointing to actual DNS of your network since with IP Forwarding enabled the DomU's will be able to access the proxy and rest of the network.
  • Now since you have enabled IP Forwarding so you should be able to ping from any other machine to you DomU's to make sure the things are working as expected you ping from C to a you should get a reply.

We are at  C and we can check

• ping 192.168.0.11 some out put like following will come

           PING 192.168.0.11 (192.168.0.11) 56(84) bytes of data.
        64 bytes from 192.168.0.11: icmp_seq=1 ttl=64 time=3.32 ms
        64 bytes from 192.168.0.11: icmp_seq=2 ttl=64 time=0.045 ms
        ^C
        --- 192.168.0.11 ping statistics ---
        2 packets transmitted, 2 received, 0% packet loss, time 1008ms
        rtt min/avg/max/mdev = 0.045/1.685/3.326/1.641 ms

• So that means that our host that is DomU1 is accessible from C and in turn from the rest of the network. Similary ssh and other things should work. If it does not happen then check if you have enabled ip forwarding. You do NOT need to enable IP FORWARDING on DomU's
  • IP Forwarding should be enabled only on Dom0.

Xen has a default network managment which will create a bridge whose name will be  same as your interface name.If you want to read more about it look at the following  Wiki page.
http://wiki.xensource.com/xenwiki/XenNetworking

        xenserver:~# brctl show
        bridge name     bridge id               STP enabled     interfaces
        eth2            8000.0026b9824238       no              peth2
                                                                vif6.0
                                                                vif7.0
                                                                vif8.0
                                                                vif9.0

• We have default ethernet as eth2 ,it can be different on your network settings.
  • If some thing on your LAN card does not work check /etc/xen/xend-config.sxp

Here is a sample note  netdev=eth2 in below output These are my entries

• vi /etc/xen/xend-config.sxp

             # Your default ethernet device is used as the outgoing interface, by default.
             # To use a different one (e.g. eth1) use
                 #
                 # (network-script 'network-bridge netdev=eth1')
                    (network-script 'network-bridge netdev=eth2')

• If (network-script 'network-bridge netdev=eth2') is commented or missing some thing

then uncomment and put the name netdev=<name of your eth interfac> as above.

• and there is one more entry you should check that is (vif-script vif-bridge) vi /etc/xen/xend-config.sxp

                # If you are using only one bridge, the vif-bridge script will discover that,
                # so there is no need to specify it explicitly.
                #
                (vif-script vif-bridge)
                If  (vif-script vif-bridge) is commented uncomment it.

• So by now your all 4 hosts must be up and running on your network and you should be able to access these DomU's from any other machine on your network.

Reverse proxy using Apache2 on Dom0 and DomU's

Section 1: Setting up of Apache2 on Dom0 and DomU

• Before you proceed to this section and do as mentioned make sure that your guest Operating Systems or DomU's are able to access your LAN.
  • We are doing this with Dom0 which has a public IP or say internet access without any proxy.
• To make apt-get work with proxy
  • mention following lines in .bashrc on individual DomU's

        export http_proxy=http://IP of your proxy:Port on which proxy runs
        export ftp_proxy=http://IP of your proxy:Port on which proxy runs

• Now its time for work aptitude install mysql-server mysql-client You will be asked to provide a password for the MySQL root user - this password is valid for the user root@localhost

 New password for the MySQL "root" user:    <-- yourrootsqlpassword Repeat password for the MySQL "root" user: <-- yourrootsqlpassword

• Apache2 is available as a Debian package, therefore we can install it like this:

 aptitude install apache2 Now direct your browser to http://192.168.0.100, and you should see the Apache2 placeholder page (It works!): aptitude install php5 libapache2-mod-php5 Repeat all these steps  for respective DomU's  so that you install apache2 on all DomU's and Dom0.

• From your LAN you can check

http://IP of Dom0/

• I created Dom0 on IP 192.168.0.100

Now open a browser and go to following url
http://192.168.0.100

• you should get a message

 It Works!! and same for all DomU's and Dom0 check
in browser and go to following urls

• http://IP of DomU1/ http://192.168.0.11/ http://IP of DomU2/ http://192.168.0.12/ http://IP of DomU3/ http://192.168.0.13/ http://IP of DomU4/ http://192.168.0.14/ you should get a message It Works!! Replace the IP of DomU with what ever you have in your network. =Section 2: Setting up for access to internet for application servers (Reverse Proxy)=
  • Now go to Dom0 and open your command prompt

        a2enmod proxy
        a2enmod proxy_http

• Now reload Apache: /etc/init.d/apache2 force-reload vi /etc/apache2/apache2.conf

        [...]
        # Include the virtual host configurations:
        Include /etc/apache2/sites-enabled/
        <Directory /var/www/>
            Options +Includes -Indexes
            AllowOverride None
            AllowOverride Indexes AuthConfig Limit FileInfo
            Order allow,deny
            Allow from all
            <Files ~ "^\.ht">
            Deny from all
            </Files>
        </Directory>
        [...]

• Right before this section, we add the following lines:

        NameVirtualHost *
        <VirtualHost *>
          ServerName xenserver.lan.in
                DocumentRoot /var/www
          ProxyRequests Off
               <Proxy *>
         Order deny,allow
               Allow from all
         </Proxy>
             ProxyPass /domu1  http://192.168.0.11/
             ProxyPass /domu2  http://192.168.0.12/
             ProxyPass /domu3  http://192.168.0.13/
             ProxyPass /domu4  http://192.168.0.14/
             ProxyPassReverse /domu1  http://192.168.0.11/
             ProxyPassReverse /domu2  http://192.168.0.12/
             ProxyPassReverse /domu3  http://192.168.0.13/
             ProxyPassReverse /domu4  http://192.168.0.14/
        </VirtualHost>

• Now you should be able to access DomUs from Internet as well.If you have a public IP on Dom0.
  • Suppose you have a domain name www.myxenserver.com So you should be able to access
    • http://www.myxenserver.com/domu1
    • http://www.myxenserver.com/domu2
    • http://www.myxenserver.com/domu3
    • http://www.myxenserver.com/domu4
• . . Then restart Apache: /etc/init.d/apache2 restart

Check points and possible errors

• The first time I booted the system Lan Card driver was not recognized. I got a error firmware: requesting bnx2-09-4.0.5.fw it is available on following page
  • http://packages.debian.org/lenny/all/firmware-bnx2/download Reboot then. If you run into any sort of problem on network check if your IP is conflicting with any other Ip on network ,ping the gateway,DNS.
  • . Open a terminal and type
  • ping <IP of Gateway>
  • ping <IP of DNS>
    • May be network card does not seems to be up then check as root
• . ifconfig -a
  • the lan card which you configured might be not the default eth0 it might be

eth1 or eth2 or eth3 like that. You need to check the LAN cable in your rack.

• Try different permutations to see that when you ping some machine on your network you
  • get a response.I was running out of space when I was creating DomU's. It may happen with you also.
• Before you choose to install separate OS as DomU's calculte the amount of RAM they will be consuming and swap space that you will be allocating them. When Dom's are running they require a certain amount of memory to be allocated to them suppose you have 8 GB RAM and you decide to give 2GB Ram to each of the virtual host you create i.e. DomU then you wont be left with memory for Dom0. We planned to give 875 Mb memory to the Dom0 so we were left with 8192-875=7317Mb of Ram which we planned to give to the DomU's.If you notice the output of xm list you can see it

     xenserver:~# xm list
                Name         ID   Mem VCPUs      State   Time(s)
                Domain-0      0   875     8     r-----    361.4
                domu1         7  3072     4     -b----     32.9
                domu2         9  1024     4     -b----     32.0
                domu3         8  1024     4     -b----     34.0
                domu4         6  2048     4     -b----     32.2

• Where domu1,domu2,domu3,domu4 are different dom's running.

References

Here are a few references that I went through in this process you may like to check them. If you want to install Xen by compiling a Xenified Debian Kernel then you should read:

• http://mightydreams.blogspot.com/2010/04/compiling-xen-from-sources-on- debian.html
• http://www.intel.com/technology/virtualization/index.htm
• http://wiki.xensource.com/xenwiki/HVM_Compatible_Motherboards
• LVM Based Set up for Xen
  • http://www.howtoforge.com/using-xen-with-lvm-based-vms-instead-of-image-based-vms- debian-etch
• Networking on Xen
  • http://wiki.xensource.com/xenwiki/XenNetworking
• Reverse Proxy on Apache2
  • http://www.howtoforge.com/apache_reverse_proxy_ispconfig
• Great tutorial about Reverse Proxy of Apache2.
  • http://www.apachetutor.org/admin/reverseproxies
• Apache module mod_proxy
  • http://httpd.apache.org/docs/1.3/mod/mod_proxy.html
• Apache2 Debian DNS Caveats,Apache2 as a proxy server
  • http://httpd.apache.org/docs/1.3/dns-caveats.html
  • http://www.debianhelp.co.uk/virtualhosts.htm
• Setting up IPTABLES Dom0 Debian Xen
  • http://permalink.gmane.org/gmane.linux.hardware.dell.poweredge/38423
• Some improvement in Xen FAQ
  • http://wiki.xensource.com/xenwiki/XenFaq
  • Squid set up http://www.freeos.com/articles/2516/
• Novell article for NAT etc on Xen
  • http://www.novell.com/communities/node/4094/xen-network-bridges-explained- with- troubleshooting-notes
• Some good information is here
  • http://lists.xensource.com/archives/html/xen-users/2006-02/msg00030.html
  • http://www.howtoforge.com/perfect_xen_setup_debian_ubuntu_p6
  • http://lists.us.dell.com/pipermail/linux-poweredge/2010-April/041936.html
  • http://wiki.xensource.com/xenwiki/Dom0
  • http://wiki.debian.org/Xen#Dom0.28host.29
  • http://wiki.xensource.com/xenwiki/XenDom0Kernels
  • http://wiki.xensource.com/xenwiki/XenParavirtOps
  • http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/snapshot_command.html
• Remote Snapshot
  • http://librenix.com/?inode=12272
  • http://community.citrix.com/display/ocb/2009/08/31/The+Xen+Cloud+Project
  • http://www.google.co.in/search?hl=en&client=firefox-a&rls=org.mozilla:en-US:official&hs=9o&q=how+to+forge+compile+Dom0+kernel+ubuntu+8.04&start=1 0&sa=N
• Installing Xen On An Ubuntu 7.10 (Gutsy Gibbon) Server From The Ubuntu Repositories - Page 2
  • http://www.howtoforge.com/ubuntu-7.10-server-install-xen-from-ubuntu-repositories-p2
  • http://deshantm.livejournal.com/29959.html?thread=24327
  • http://www.howtoforge.com/installing-xen-3.3-with-kernel-2.6.27-on-ubuntu-8.10-x86_64-p2
  • http://howtoforge.com/ubuntu-8.04-server-install-xen-from-ubuntu-repositories-p2
  • http://www.howtoforge.org/installing-xen-3.3-with-kernel-2.6.27-on-ubuntu- 8.10- x86_64
  • http://ubuntuforums.org/showthread.php?p=8403877#post8403877
  • http://www.infohit.net/blog/post/deploying-domu-centos-5-with-xen-on- ubuntu-904- jaunty-jakalope.html
  • https://help.ubuntu.com/community/Xen
• CentOS DomU on Ubuntu Hardy
  • http://www.howtoforge.com/create-centos5.2-domu-on-ubuntu-hardy-dom0
  • http://www.linuxinstead.com/blog/?p=63#more-63
  • http://www.xen.org/support/ http://wiki.debian.org/Xen
  • http://wiki.xensource.com/xenwiki/XenDom0Kernels/
  • http://ganto.no-ip.org/?cat=20
• Then some more
  • http://librenix.com/?page=Xen
  • http://www.gamelinux.org/?cat=4
• Installing Xen 3.0 upon Debian Unstable, with a custom Kernel
  • http://www.debian-administration.org/articles/320
• Some more help
  • http://cwshep.blogspot.com/2008/12/howto-ubuntu-intrepid-ibex-810-xen- dom0.html
• Re: Debian testing + xen 3.2.1 + dom0 kernel 2.6.18 + clvm 2.x -> impossible ?
  • http://lists.debian.org/debian-user/2008/09/msg01745.html
  • http://gnutolinux.wordpress.com/2009/02/17/debian-50lennyreleased/
• Has got mentioned using ATA over ethernet
  • http://www.unix-tutorials.com/search.php? act=search&term=How+To+Compile+A+Kernel+-+Debian+Etch&page=25 Installing a Kernel and Generating a Menu List
  • http://library.linode.com/advanced/pv-grub-howto
• Fedora Core 8 as DomU
  • http://www.ilande.co.uk/index.php?/categories/2-Blog
• An interesting article related to Xen http://www.debian-administration.org/articles/505
  • http://pdg86.wordpress.com/2009/09/16/howto-compile-2-6-31-kernel-in-slackware-13-0/
• Compile a Dom0 kernel the Debian way http://lists-archives.org/debian-user/3481939-how-to-compile-a-xen-dom0-kernel-the- debian-way.html
  • http://swik.net/Debian+virtualization
• http://readlist.com/lists/lists.xensource.com/xen-users/4/20551.html
• This is a very good link to study http://librenix.com/?find=Boris%20Derzhavets
  • http://lists.zerezo.com/debian-user/msg3494236.html
• Some tutorials on this page
  • http://www.unix-tutorials.com/search.php? act=search&term=Debian+Kernel+Compile+Howto+(Kernel+2.6)&page=29 Building custom kernel packages
  • http://wiki.debian.org/HowToRebuildAnOfficialDebianKernelPackage
• They discuss here about closing the 2.6.18 tree
  • http://lists.xensource.com/archives/html/xen-devel/2009-06/msg00241.html
• This is the discussion going on about Virtualization where they have talked relevant things about patches etc http://lists.xensource.com/archives/html/xen-devel/2009-06/msg00127.html
  • http://en.opensuse.org/Xen_Full_Virtualization_Example
  • http://www.cl.cam.ac.uk/research/srg/netos/xen/documentation.html
  • http://lists.xensource.com/archives/html/xen-users/2006-07/msg00100.html
  • http://wiki.xensource.com/xenwiki/HVM_Compatible_Processors
• Intel http://wiki.xensource.com/xenwiki/IntelVT
• A step by step guide and some relevant discussion
  • http://lists.xensource.com/archives/html/xen-users/2006-05/msg00568.html
• Xen Full Virtualization examples
  • OpenSuse http://en.opensuse.org/Xen_Full_Virtualization_Example
• Remote VNC Solutions http://help.punt.nl/?id=421411&r=1
• Some Faq about Xen http://wiki.xensource.com/xenwiki/XenFaq#head- 31bd14b4c509f5e23c4f82250273ac642364a060 Xen Related entire thing from Cambridge
  • http://www.cl.cam.ac.uk/research/srg/netos/xen/readmes/interface/interface.html
• FaQ's about mercurial
  • http://mercurial.selenic.com/wiki/FAQ#General_Questions
  • Some information regarding Mercurial http://www.cl.cam.ac.uk/research/srg/netos/xen/readmes/hg-cheatsheet.txt
  • http://www.intel.com/technology/virtualization/index.htm
  • http://wiki.xensource.com/xenwiki/HVM_Compatible_Motherboards
• All the commands for virtualization
  • http://centos.org/docs/5/html/5.2/Virtualization/chap-Virtualization- Managing_guests_with_virsh.html
• Stress testing of Virtualization Server is mentioned here
  • http://centos.org/docs/5/html/5.2/Virtualization/part-Virtualization-Installation_Procedures.html
• Guest and Host OS compatibility list here
  • http://centos.org/docs/5/html/5.2/Virtualization/chap-Virtualization- Compatibility_of_host_and_guest_combinations.html
• Blktap driver documentation
  • http://lxr.xensource.com/lxr/source/tools/blktap/
  • http://wiki.centos.org/HowTos/Xen/MoveNative2DomU
  • http://wiki.xensource.com/xenwiki/Dom0
  • http://wiki.debian.org/Xen#Dom0.28host.29
  • http://wiki.xensource.com/xenwiki/XenDom0Kernels
  • http://wiki.xensource.com/xenwiki/XenParavirtOps
  • http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/snapshot_command.html
• Remote Snapshot
  • http://librenix.com/?inode=12272
  • http://community.citrix.com/display/ocb/2009/08/31/The+Xen+Cloud+Project
  • http://www.google.co.in/search?hl=en&client=firefox-a&rls=org.mozilla:en-S:official&hs=9o&q=how+to+forge+compile+Dom0+kernel+ubuntu+8.04&start=10&sa=N
• Installing Xen On An Ubuntu 7.10 (Gutsy Gibbon) Server From The Ubuntu Repositories - Page 2
  • http://www.howtoforge.com/ubuntu-7.10-server-install-xen-from-ubuntu-repositories-p2
  • http://deshantm.livejournal.com/29959.html?thread=24327
  • http://www.howtoforge.com/installing-xen-3.3-with-kernel-2.6.27-on-ubuntu-8.10-x86_64-p2
  • http://howtoforge.com/ubuntu-8.04-server-install-xen-from-ubuntu-repositories-p2
  • http://www.howtoforge.org/installing-xen-3.3-with-kernel-2.6.27-on-ubuntu-8.10-x86_64
  • https://help.ubuntu.com/community/Xen
• CentOS DomU on Ubuntu Hardy
  • http://www.howtoforge.com/create-centos5.2-domu-on-ubuntu-hardy-dom0
  • http://www.xen.org/support/
  • http://wiki.debian.org/Xen
  • http://wiki.xensource.com/xenwiki/XenDom0Kernels/
  • http://ganto.no-ip.org/?cat=20
• Installing Xen 3.0 upon Debian Unstable, with a custom Kernel
  • http://www.debian-administration.org/articles/320
• Installing a Kernel and Generating a Menu List
  • http://library.linode.com/advanced/pv-grub-howto
• Fedora Core 8 as DomU
  • http://www.ilande.co.uk/index.php?/categories/2-Blog
• An interesting article related to Xen
  • http://www.debian-administration.org/articles/505
  • http://pdg86.wordpress.com/2009/09/16/howto-compile-2-6-31-kernel-in-slackware-13-0/
• Compile a Dom0 kernel the Debian way
  • http://lists-archives.org/debian-user/3481939-how-to-compile-a-xen-dom0-kernel-the- debian-way.html
  • http://mercurial.selenic.com/wiki/FAQ#General_Questions
• Relevant tutorials from IBM
  • http://www.ibm.com/developerworks/linux/
  • http://www.ibm.com/developerworks/linux/library/l-pci-passthrough/index.html? S_TACT=105AGX03&S_CMP=HP
  • http://www.ibm.com/developerworks/linux/library/l-top-10.html? S_TACT=105AGX03&S_CMP=HP
• 7 Features to consider before hypervisor installation
  • http://searchservervirtualization.techtarget.com/tip/0,289483,sid94_ gci1375660_mem1,00.ht ml?asrc=SS_CLA_304683&psrc=CLT_94
• Building a Xen Virtual Guest Filesystem on a Disk Image (Cloning Host System)
  • http://www.virtuatopia.com/index.php/Building_a_Xen_Virtual_Guest_Fi lesystem_on_a_Di sk_Image_(Cloning_Host_System)
• Red Hat System Administration Guide
  • http://web.mit.edu/rhel-doc/4/RH-DOCS/index-en.html
  • http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/security-guide/s1-firewall- state.html
• Building a Xen Virtual Guest Filesystem on a Disk Image (Cloning Host System)
  • http://www.virtuatopia.com/index.php/Building_a_Xen_Virtual_Guest_Filesystem_on_a_Disk_Image_(Cloning_Host_System)
• Virtualization Saves Microsoft Customers Nearly a Half-Million Dollars Per Year
  • http://www.microsoft.com/presspass/press/2009/feb09/02- 09virtualizationsavingspr.mspx
• Installing Windows DomU on Xen
  • http://lists.xensource.com/archives/html/xen-users/2006-06/msg00599.html
• Overview of Virtualization
• http://www.virtuatopia.com/index.php/An_Overview_of_Virtualization_Techniques
  • Happy Xenning!!!

If you encounter any problem doing above or want to give some suggestion that can be added in this page

• drop me a mail Tapas: mightydreams@gmail.com