Book/HelloXenProject/2-Chapter

Chapter 2 : Xen Project

What is XenProject?

Welcome to chapter 2! In this chapter, we will talk about Xen Project more and cover Installation Xen Project. OK, Let's Start!

The Xen Project Hypervisor is an Open Source “Type-1” or Bare Metal Hypervisor which let you run many OS on your server. Operating systems on your server will run in parallel mode. Don't forget, when writing this book, The Xen Project is the only type-1 hypervisor that is Open Source. As we said, some companies like Citrix distribute Xen as a commercial product like “Citrix XenServer” but the range of Xen use is not limited to it and this project used in many areas like Desktop Virtualization, Server Virtualization, Cloud computing and…

Xen Project has tons of features, but we just consider some of the most important of them:

• Xen uses Microkernel design and according to the Xen Project Wiki it just 1MB in size but have a good security.
• Flexibility in Driver: By default, Linux Driver domains for launch back ends for guests depend on udev and it has some problems. For example, If driver crashed or infected, then the driver can be rebooted without affecting the rest of the system. In Xen 4.4, udev replaced with a custom daemon that built top of libxl and provide a good flexibility.
• Paravirtualization: We spoke about it and said that the Paravirtualization introduced by Xen Project. This allows your guest to run much faster and our Hypervisor can run on hardware that don't support Virtualization extension.
• Nested: The Xen Nested provides Hardware Virtualization to Guests and via this ability you can run Xen Project, KVM, Hyper-V and VMware ESX inside a guest and launching your test environment. The Hypervisor that run on real Hardware is called “Level 0” and the Hypervisor that run on Guest called “Level 1” and a guest that running via “Level 1” called “Level 2”.
• Better support of Virtual Desktops Protocols (SPICE). Via SPICE you can have a better Remote Desktop compared with VNC.
• A good news is that the Xen Project supports Grub 2 and don't need any custom implementation of Grub (PvGrub). In the past, The Xen project used “PyGrub” and its behave like Grub and read the standard grub file “menu.lst “ to provide the Xen Project for creating process. The PvGrub is a more efficient and secure alternative for “PyGrub”, But nowadays, The Xen project support “Grub” and it is a good feature.
• XAPI and Mirage OS: Both of them are sub-projects of Xen Project that written in Ocaml programming language. The Mirage is a Cloud OS for making secure and high performance network applications that using on Cloud computing, Embedded devices and Mobile platforms. The XAPI is another project that enables Power saving, cooling and management cost and let you to use your hardware in optimize mode and improve reliability.
• Guest EFI boot: EFI is a new booting standard that used instead of BIOS and nowadays some OS just uses EFI to boot. The Xen Project can support it and can be booted from EFI platform and bootloader like Grub.
• Support of ARM processor

They were some of the most Xen Project features and I bet you can find other important features too, but since our book is a short and practical book then we can't consider and dive into all of them.

In a below texts, I want to show you something about the Xen Project Architecture and I copy a good Figure from Xen Project web site :

Figure 9 : The Xen Project Architecture

A simple view of an old Xen diagram is as below:

Figure 10 : A simple overview of Xen Architecture

The Xen Hypervisor run on the hardware directly and managing hardware components like CPU, Memory, Storage and…. A VM called “Domain” or “Guest” and a special domain that belonging to Xen is “Domain 0” and contains of all Hardware drivers. We can say “Domain 0” is a special Virtual Machine too.

In the above diagram, you can see “Xend” that is discontinued and removed in Xen 4.5. It was a node control daemon that perform system management that related to Virtual Machines and control the Virtual Resources. The new replacement for “Xend” is “xl”. It is true that “xl” introduced in Xen 4.1 but “Xend” remain as default and with the advent of Xen 4.5 the “Xend” removed and give its place to “xl”.

The “xl” have better features compared with “Xend” and you can find a list of these features in below link: XL_vs_Xend_Feature_Comparison

We said something about “Xend” and in Xen 2.X, The Toolstack actually used for managing Xen installations. In Xen 4.1, a new Toolstack called libxenlight that also know as libxl. The libxl is a Small and Low level Library that written in C language and easy to understand and modify. It has been designed to provide a simple API for all client Toolstacks. It is a good feature of Xen Project. One of the big differences between xend and libxl is that the “Xend” is stateful while libxl is stateless. With “Xend”, all client applications such as xm and libvirt see the same system state, but In libxl, client applications such as xl or libvirt must maintain state.

If we want wrap-up libxl features then they are:* Stateless

• So simple
• Implement mechanisms, not policies
• Hide components like xenstore, libxenctrl, and libxenguest from higher levels

In the Xen Project, Each Toolstack provides an API with different tools. We mentioned XAPI before, which is exclusively used by XenServer (see xenserver.org and exposes more functionality than the default toolstack (libxl) and libvirt in that it allows managing VMs across multiple hosts and adds abilities like management of pools of host systems, support for advanced storage repositories, good support of Open vSwitch and guaranteed SLAs. XAPI today is only available from xenserver.org: however historically there has been a variant called XCP. If you want to find out more about XAPI and XenServer, please go to xenserver.org.

In the figure below you will see different example Xen Toolstacks for common Xen Toolstacks:

Figure 11 : Xen Tool Stack

Note that other Xern based solutions such as https://www.qubes-os.org/ or http://openxt.org/ use their own toolstack implementation.

How does Paravirtualization work?

Time has come for talk about Paravirtualization.

If you remember, Paravirtualization (PV) was introduced by Xen Project. PV does not require Virtualization Extension from CPU and enable Virtualization on Hardware that don't support Hardware Virtualization, But require Kernel support and special driver and Kernel built that nowadays are part of The Linux kernel and other OS.

PV is a new term for an old idea from IBM. In other hand, When Xen Hypervisor running on Hardware that doesn't support Hardware Virtualization then Xen need that the guest OS customized. This customized OS using an abstract hardware model that differs from the specific hardware that is available on the physical machine and can execute at lower privilege level than the hypervisor. As you know, The Hypervisor handle CPU, Memory, System calls and… and this technique known as “Paravirtualization” because our modified OS in order to handle privileged operations must communicate with the hypervisor that administrated by Xen.

As you know some Operating Systems like Microsoft Windows is Closed source and the Xen Project windows provide a kit of paravirtualization device drivers under GPL license that installed on Windows OS. I guess you understood that in PV, The Hypervisor act like a layer that control access of guest OS to the underlying hardware resource.

Why PV? Because it has some advantage over other techniques. Performance is the most obvious advantage because it is a very small amount of code and as we said PV is layered and can act like a traffic controller too. For example, Allowing one OS to have access to physical Hardware while stopping other OS from accessing the same resource. Another good feature is that PV shows you an unlimited way about device drivers in the guest OS. What do you think about Disadvantage? I know that your thinking about Closed Source OS like Microsoft Windows. You know that PV need the guest operating systems be modified in order Paravirtualization, but how? Fortunately, Citrix has provided a set of PV driver for Windows from XenServer product and these drivers become Open Source under BSD license. The team from Xen Project that working on Windows PV Drivers, maintaining and developing these drivers under Xen Project governance. These drivers can be found at “http://www.xenproject.org/downloads/windows-pv-drivers.html”. Again, As you remember, The Intel and AMD CPUs provide functionality that enables unmodified operating systems to be hosted by a paravirtualized hypervisor. In addition to the Citrix and other companies like Novell that provided PV Drivers, Some GPLed drivers produced by developers like “James Harper”. For more information, Please see “http://wiki.univention.de/index.php?title=Installing-signed-GPLPV-drivers”. As you see, All people Love Xen.

OK, Speaking about Xen is enough and Let me jump to the Xen installing.

How to Install Xen project?

For installing Xen Project, You have two options : # Install via your Linux Distro package manager

1. Install from source code.

First of all, Let me guess, what do you think! I guess you are thinking that which Linux Distro is better for Xen. I want to tell you that it is a personal preference and depended on your skill. If you have an experience in Fedora then Fedora is good for you and if you have a background with Debian then Debian is good, But Distros like Debian, Ubuntu, openSUSE, SUSE Linux Enterprise Server, Fedora, and NetBSD have a good support for Xen Project and if you want select CentOS then you must consider that CentOS needs extra software for it. For an update list about which distro is OK with Xen Project, Please look at the Dom0_Kernels_for_Xen URL. I choose Fedora for as Dom0 and I consider that you installed Fedora on your server. If you don't know how you can install Linux then Please Google “how to install Fedora”. I choose “Fedora Server Edition” and when you want to install it, I recommended to select “Headless Virtualization”:

Figure 12 : Fedora Server Edition

Welcome to Fedora, by default when you install Fedora, It will install some packages like “LibreOffice” but it depended to your install scenario. If you install Fedora Workstation then it installed “LibreOffice” and we don't like to update “LibreOffice”. If you like all of your Fedora components updated then just run “yum update” else follow me.

I don't like all of my Fedora components like “LibreOffice” updated because it is a server and I don't like to write and edit “.odt”, “.doc” and “.docx” files on it, So I want to exclude “LibreOffice” from the update. To exclude a package from update you should edit “yum.conf” file as below: 1- open “yum. conf” file from “/etc/yum.conf” with an editor like “vim” or “nano”(In newer version of Fedora it changed to “dnf. conf” and is under “ /etc/dnf/dnf.conf”).

In Fedora you can use “dnf” instead of “yum” and for more information see “http://dnf.readthedocs.org/en/latest/command_ref.html”. For example :

# yum update == # dnf update

2- use “exclude=” for ignoring specific packages. For example, for ignoring “LibreOffice” package adds “exclude=libreoffice*” and the star at the end of “LibreOffice” mean all packages with start “LibreOffice”. My “dnf.conf” file is as below :

[main]

gpgcheck=1

installonly_limit=3

clean_requirements_on_remove=True

exclude= LibreOffice *

Notice that you can use “--exclude=” parameter in command line too. For example :

# dnf update --exclude= LibreOffice *

3- Run “dnf update” as root user or via “sudo” command.

# dnf update

4- After an update, The time has come to install “Xen”, But you must consider that for the Xen Project on Fedora you must add a repository. It is not mandatory, but In order to enable this repository run below commands:

# cd /etc/yum.repos.d/

# wget http://fedorapeople.org/groups/virt/virt-preview/fedora-virt-preview.repo

# dnf update

5- You can install Xen Project now, but if you have any issues then you should Disable “Selinux”. The Selinux problems with Xen on Fedora 21 and later are solved, but for disable Selinux you can do below steps:

First of all, check Selinux status by a run “/usr/sbin/sestatus” :

[root@mohsen ~]# sestatus

SELinux status:  enabled

SELinuxfs mount:  /sys/fs/selinux

SELinux root directory:  /etc/selinux

Loaded policy name:  targeted

Current mode:  enforcing

Mode from config file:  enforcing

Policy MLS status:  enabled

Policy deny_unknown status:  allowed

Max kernel policy version:  29

For disable Selinux open “/etc/selinux/config” via nano and then change “SELINUX=enforcing” to “SELINUX=disabled” :

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.

SELINUX=disabled

# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.

SELINUXTYPE=targeted

After it you are ready to install the Xen Project :

# dnf install xen

After that the Xen project installed successfully, then you must configure Grub. You below command for configure your grub:

# grub2-mkconfig -o /boot/grub2/grub.cfg

and

# reboot

When your system rebooted, then you can see other options on your grub menu:

Figure 13 : Grub with Xen Hypervisor

By default grub use the first option to boot, but if you want your system boot via Xen Hypervisor always then you can change this default option via below commands:

# grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2

Figure 14 : Grub Entry

# grub2-set-default “Fedora, with Xen hypervisor”
# grub2-editenv list
# grub2-mkconfig -o /boot/grub2/grub.cfg

After it, You can see that “ Fedora, with Xen hypervisor” is selected by default.

OK, We boot Fedora via Xen Project and run below command to sure that The Xen Project installed successfully :

[root@localhost ~]# xl info

host : localhost.localdomain

release : 4.4.7-300.fc23.x86_64

version : #1 SMP Wed Apr 13 02:52:52 UTC 2016

machine : x86_64

nr_cpus : 8

max_cpu_id : 7

nr_nodes : 1

cores_per_socket : 4

threads_per_core : 2

cpu_mhz : 3392

hw_caps : bfebfbff:28100800:00000000:00003f00:179ae3bf:00000000:00000001:00000000

virt_caps : hvm

total_memory : 8109

free_memory : 128

sharing_freed_memory : 0

sharing_used_memory : 0

outstanding_claims : 0

free_cpus : 0

xen_major : 4

xen_minor : 5

xen_extra : .3

xen_version : 4.5.3

xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64

xen_scheduler : credit

xen_pagesize : 4096

platform_params : virt_start=0xffff800000000000

xen_changeset :

xen_commandline : placeholder

cc_compiler : gcc (GCC) 5.3.1 20151207 (Red Hat 5.3.1-2)

cc_compile_by : mockbuild

cc_compile_domain : [unknown]

cc_compile_date : Wed Mar 30 19:19:01 UTC 2016

xend_config_format : 4

If you got any error like below :

# xl info

xc: error: Could not obtain handle on privileged command interface (2 = No such file or directory): Internal error)

libxl: error: libxl.c:114:libxl_ctx_alloc: cannot open libxc handle: No such file or directory cannot init xl context

Then you must consider using commands like “systemctl enable xend.service” and “systemctl enable xendomains.service” and retry. The Xen Project has a good community and you can ask your questions from them.

Now that we have installed Xen and boot via its successfully, then we must cover something about the Xen itself.

Xen Project Compatibility

I guess compatibility is a big problem for you to use Xen. For example, You may ask yourself that is Xen Project compatible with my Hardware? Can I use Xen Project on my current Hardware? And etc.

You can run Xen on many hardware flavors like Laptops and you should not have any worry about it. The Xen project, Can support many architectures and if you see a table that we introduced it in Chapter one then you can understand it. Currently, The Xen project can support x86, x86_64 and ARM architectures. Another question that you mas ask is your VGA. Can the Xen Project support my Nvidia or ATI VGA card? The Xen Project has a good community about VGA adapters that you can find it at Xen_VGA_Passthrough_Tested_Adapters and in this URL you can find a list of VGA models that tested by the community. By the way, at “https://wiki.centos.org/HowTos/Xen/NvidiaWithXen” URL you can see that some people installed Nvidia driver on Xen and it worked properly. As I said, Some OS like “qubes-os” use Xen and according to the “qubes-os” documents you can install Nvidia Driver with some tricks. For more information, Please see “https://www.qubes-os.org/doc/install-nvidia-driver/”. Citrix XenServer at “http://hcl.xenserver.org/” provide a good list of The Hardware and servers that supported by Citrix XenServer.

You can see your CPU features in Linux by below command :

# cat /proc/cpuinfo

And for mine it is something like :

processor: 0

vendor_id: GenuineIntel

cpu family: 6

model: 42

model name: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz

stepping: 7

microcode: 0x17

cpu MHz: 3392.414

cache size: 8192 KB

physical id: 0

siblings: 8

core id: 0

cpu cores: 4

apicid: 0

initial apicid: 0

fpu: yes

fpu_exception: yes

cpuid level: 13

wp: yes

flags: fpu de tsc msr pae mce cx8 apic sep mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl nonstop_tsc eagerfpu pni pclmulqdq monitor est ssse3 cx16 sse4_1 sse4_2 popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm ida arat epb pln pts dtherm xsaveopt

bugs:

…

I guess it is enough and main features are that The Xen Project can support popular architectures and Hardware. Let me to dive into the Xen Project components.

Xen Project Components

The Xen Project components consist of the Xen hypervisor, the Domain0 and DomU that is any number of VM. A Physical computer running all of these components that referred to it as a VM Host.

We spoke something about these components, but we do a short reminder:

The Xen Hypervisor:

It is a Virtual Machine Monitor that is an Open Source software that coordinate Low Level parts between Vms and Physical Hardware.

The Domain0:

The Domain0 is controlling domain that comprised of Host OS, The Xend daemon and a modified version of Qemu.

The VM Guest or DomU:

The DomU consist of Virtual Disk, Network Devices and other Hardware Plus Virtual Machine configuration files.

Figure 15 : Another view of The Xen Project Components.

Xen Project Commands

OK, Now is a good time for speaking about Xen commands and if you remember we said that “xm” is removed from Xen version 4.5, Thus we will cover “xl” here. We will not cover all parameters of “xl” here, but we will most important of it.

You are as a Linux administrator familiar with “man” command and for finding a full details of “xl” command and parameters Please type “man xl” in your console. I prepared a table of most important commands that you can see it below :

CommandDescription

xl helpView available options and help.

xl listShow active domains.

xl createStart and create a VM.

xl rebootReboot a VM.

xl shutdownShutdown a VM.

xl destroyTerminate a VM immediately. It is like Shutdown but work immediately.

xl consoleAttach to a VM console.

xl infoShow information about Xen Host.

xl topMonitor host and domain in real time.

xl network-listList of Virtual network interfaces.

xl dmesgRead or clear dmesg buffer.

xl pausePause A domain.

xl unpauseUnpause a domain.

xl saveSave a domain state.

xl vcpu-listList the VCPUs for all/some domains.

xl uptimeShow uptime for all/some domains

They are not all parameters and I'm sure you can find other useful parameters too. Just run “xl help” and you will find a list of all available parameters. We can't cover all of the here and we just show you some of them.

Compiling Xen from Source Code

I guess you are thinking that about installing The Xen Project from Source Code and we haven't forgotten it. To be frank, When your Linux Distro has package management and The Xen Project prepared as a package for your Linux Distro then most users (Administrators and novice) use it and don't like to compile it from Source Code unless you like to use special parameters when installing The Xen Project.

The Xen Project Wiki prepares a section about this goal and you can find useful information about it in Compiling_Xen_From_Source URL.