Xen Project Software Overview

What is Xen?

Xen is an open-source type-1 or baremetal hypervisor, which makes it possible to run many instances of an operating system or indeed different operating systems in parallel on a single machine (or host). Xen is the only type-1 hypervisor that is available as open source. Xen is used as the basis for a number of different commercial and open source applications, such as: server virtualization, Infrastructure as a Service (IaaS), desktop virtualization, security applications, embedded and hardware appliances. Xen is powering the largest clouds in production today.

Here are some of Xen's key features:

• Small footprint and interface (is around 1MB in size). Because Xen uses a microkernel design, with a small memory footprint and limited interface to the guest, it is more robust and secure than other hypervisors.
• Operating system agnostic: Most installations run with Linux as the main control stack (aka "domain 0"). But a number of other operating systems can be used instead, including NetBSD and OpenSolaris.
• Driver Isolation: Xen has the capability to allow the main device driver for a system to run inside of a virtual machine. If the driver crashes, or is compromised, the VM containing the driver can be rebooted and the driver restarted without affecting the rest of the system.
• Paravirtualization: Fully paravirtualized guests have been optimized to run as a virtual machine. This allows the guests to run much faster than with hardware extensions (HVM). Additionally, Xen can run on hardware that doesn't support virtualization extensions.

This page will explore the key aspects of Xen architecture that a user is likely to understand in order to make the best choices about Xen.

• Guest types: Xen can run fully virtualized (HVM) guests, or paravirtualized (PV) guests.
• Domain 0: Xen has a special domain called domain 0 which contains drivers for the hardware, as well as the toolstack to control VMs.
• Toolstacks: This section covers various toolstack front-ends available tor Xen and the implications of using each.

Introduction to Xen Architecture

Below is a diagram of the Xen architecture. The Xen hypervisor runs directly on the hardware and is responsible for handling CPU, Memory, and interrupts. It is the first program running after exiting the bootloader. On top of Xen run a number of virtual machines. A running instance of a virtual machine in Xen is called a domain or guest. A special domain, called domain 0 contains the drivers for all the devices in the system. Domain 0 also contains a control stack to manage virtual machine creation, destruction, and configuration.

Components in detail:

• The Xen Hypervisor is an exceptionally lean (<150,000 lines of code) software layer that runs directly on the hardware and is responsible for managing CPU, memory, and interrupts. It is the first program running after the bootloader exits. The hypervisor itself has no knowledge of I/O functions such as networking and storage.
• Guest Domains/Virtual Machines are virtualized environments, each running their own operating system and applications. Xen supports two different virtualization modes: Paravirtualization (PV) and Hardware-assisted or Full Virtualization (HVM). Both guest types can be used at the same time on a single Xen system. It is also possible to use techniques used for Paravirtualization in an HVM guest: essentially creating a continuum between PV and HVM. This approach is called PV on HVM. Xen guests are totally isolated from the hardware: in other words, they have no privilege to access hardware or I/O functionality. Thus, they are also called unprivileged domain (or DomU).
• The Control Domain (or Domain 0) is a specialized Virtual Machine that has special privileges like the capability to access the hardware directly, handles all access to the system’s I/O functions and interacts with the other Virtual Machines. It also exposes a control interface to the outside world, through which the system is controlled. The Xen hypervisor is not usable without Domain 0, which is the first VM started by the system.
• Toolstack and Console: Domain 0 contains a control stack (also called Toolstack) that allows a user to manage virtual machine creation, destruction, and configuration. The toolstack exposes an interface that is either driven by a command line console, by a graphical interface or by a cloud orchestration stack such as OpenStack or CloudStack.
• Xen-enabled operating systems: A Xen Domain 0 requires a Xen-enabled kernel. Paravirtualized guests require a PV-enabled kernel. Linux distributions that are based on recent Linux kernel are Xen-enabled and usually contain packages that contain the Xen Hypervisor and Tools Xen (the default Toolstack and Console). All but legacy Linux kernels are PV-enabled: in other words, they will run Xen PV guests.

Also see:

• Xen Features
• Xen-Enabled operating systems
• PV-Enabled operating systems
• Availability of Xen Functionality on Linux Kernel (by version)

Guest Types

Xen supports running two different types of guests: Paravirtualization (PV) and Full or Hardware assisted Virtualization (HVM). Both guest types can be used at the same time on a single Xen system. It is also possible to use techniques used for Paravirtualization in an HVM guest: essentially creating a continuum between PV and HVM. This approach is called PV on HVM.

Also see:

• Virtualization Spectrum

Xen Paravirtualization (PV)

Paravirtualization is an efficient and lightweight virtualization technique introduced by Xen, later adopted by other virtualization platforms. PV does not require virtualization extensions from the host CPU. However, paravirtualized guests require a Xen-PV-enabled kernel and PV drivers, so the guests are aware of the hypervisor and can run efficiently without emulation or virtual emulated hardware. Xen-PV-enabled kernels exist for Linux, NetBSD, FreeBSD and OpenSolaris. Linux kernels have been Xen-PV enabled from 2.6.24 using the Linux pvops framework. In practice this means that PV will work with most Linux distributions (with the exception of very old versions of distros).

Also see:

• More Information...

Xen Full Virtualization (HVM)

Full Virtualization or Hardware-assisted virtualizion uses virtualization extensions from the host CPU to virtualize guests. HVM requires Intel VT or AMD-V hardware extensions. Xen uses Qemu to emulate PC hardware, including BIOS, IDE disk controller, VGA graphic adapter, USB controller, network adapter etc. Virtualization hardware extensions are used to boost performance of the emulation. Fully virtualized guests do not require any kernel support. This means that Windows operating systems can be used as Xen HVM guest. Fully virtualized guests are usually slower than paravirtualized guests, because of the required emulation.

PV on HVM

To boost performance, fully virtualized HVM guests can use special paravirtual device drivers (PVHVM or PV-on-HVM drivers). These drivers are optimized PV drivers for HVM environments and bypass the emulation for disk and network IO, thus giving you PV like (or better) performance on HVM systems. This means that you can get optimal performance on guests operating systems such as Windows.

Note that Xen PV (paravirtual) guests automatically use PV drivers: there is thus no need for these drivers - you are already automatically using the optimized drivers. PVHVM drivers are only required for Xen HVM (fully virtualized) guest VMs.

Also see:

• More Information...
• Information about using PV-on-HVM drivers
• HowTo on PV-on-HVM drivers
• Xen Windows HVM guests can use the opensource GPLPV drivers

PV in an HVM Container (PVH) - New in Xen 4.3

Xen 4.3 will bring a virtualization mode called PVH. This is essentially a PV guest using PV drivers for boot and I/O. Otherwise it uses HW virtualization extensions, without the need for emulation. First patches are going into xen-unstable just after the Xen 4.2 release and functionality should be at least available as a pre-view in Xen 4.3. PVH has the potential to combine the best trade-offs of all virtualization modes, while simplifying the Xen architecture.

For more info see:

• PVH: PV Guest in HVM container (presentation)
• Xen on ARM Cortex A15 (presentation) - also uses PVH
• PVH: PV Guest in HVM container (video)
• Xen on ARM Cortex A15 (video) - also uses PVH

Toolstacks, Managment APIs and Consoles

Xen can run with a number of different toolstacks. Each toolstack exposes an API, which will run different tools. The figure below gives a very brief overview of the choices you have, which commercial products use which stack and examples of hosting vendors using specific APIs.

In short: Xen can be run with the default toolstack, with Libvirt and with XAPI. When Xen is run with XAPI, we tend to call the resulting stack XCP. The diagram above shows the various options: all of them have different trade-offs and are optimized for different use-cases. However in general, the more on the right of the picture you are, the more functionality will be on offer.

Which to Choose?

• The article Choice of ToolStacks gives you an overview of the various options, with further links to tooling and stacks for a specific API exposed by that toolstack.
• Xen or XCP also provides good pointers on whether to use Xen or XCP.

Of course there are also additional management tools available for different API's. For more information see:

Xen:

• Xen Management Tools
• Open Source Tools and Software interfacing with Xen
• Commercial Tools and Software interfacing with Xen

XCP:

• XCP Management Tools
• Open Source Tools and Software interfacing with XCP
• Commercial Tools and Software interfacing with XCP

Getting Xen, Host and Guest Install

Choice of Control Domain (Dom0)

As stated earlier, Xen requires a kernel as control domain. Most Xen-enabled kernels are very similar from the perspective of Xen. Choosing the right Dom0 for you comes down to:

• How familiar you are with a specific distro (e.g. packaging system, etc.)
• Xen Hypervisor version that ships with the distro
• Whether you can get commercial support (if you need it)

If you use XCP, you typically will not be interfacing much with Dom0. That is unless you are a power user.

Also See

• Dom 0 Kernels
• Getting Started

Getting Xen

The Xen hypervisor is available as source distribution from Xen.org. However, you can get recent Xen binaries as packages from many Linux and Unix distributions, both open source and commercial. The Xen Cloud Platform ISOs also contain the Xen Hypervisor in binary form.

Xen Source Distributions The Xen community delivers the Xen hypervisor as a source distribution, following the delivery model of the Linux kernel. Xen is released approximately once a year, with several update releases per year containing security fixes and critical bug fixes. To build Xen from source, you can either download a Xen source release or you can fetch the source tree from the source repository. Each Xen source release and the Xen source tree contain a README file in the root directory, with detailed build instructions for the Xen hypervisor. The release notes for Xen distributions also contain build instructions and so does the Compiling Xen page.

Xen Cloud Platform The Xen Cloud Platform (or XCP) is a binary distribution of Xen that contains the Xen Hypervisor, a pre-packaged and configured Control Domain and the powerful Xen API toolstack and management API. XCP can be installed from either a single installable ISO, or by running

apt-get install xcp-xapi

on a Debian-based Linux distribution.

XCP delivers an enterprise-ready, server virtualization and cloud computing platform with lots of additional management functionality compared Xen: with XCP you can manage pools of host systems, use advanced storage repositories, and take advantage of advanced performance monitoring capabilities. Download here!

Xen in Linux/Unix Distributions Most Linux and many Unix distributions contain built binaries of Xen that can be downloaded and installed through the native package management system. If your Linux/Unix distribution includes Xen and a Xen-enabled kernel, we recommend to use them as you will benefit from ease of install, good integration with the distribution, support from the distribution, provision of security updates etc. Installing Xen in a distribution typically requires the following basic steps: a) Install your favourite distribution, b) Install xen package(s) or meta-package, c) check boot settings and d) reboot. After the reboot, your system will run your favourite Linux/Unix distribution as Control Domain on top of the Xen Hypervisor.

Host and Guest Install

The following documents

• Category:Host Install contains guides on how to install a control domain for various Linux/Unix distros
• Category:Guest Install contains guides on how to install a variety of guests for various Linux/Unix distros

This table contains a list of Xen Project resources for various Linux and Unix distributions.

[edit]

Distro	Main website	Description	Resources
Arch Linux	archlinux.org	Arch Linux is a lightweight and flexible Linux® distribution that tries to “keep it simple”.	Arch as Xen Host Arch as Xen Guest (PV) All Arch_Linux pages
Alpine Linux	alpinelinux.org	A security-oriented, lightweight Linux distribution based on musl libc and busybox.	Xen Dom0 on a HDD Xen Dom0 on USB / SDD Xen DomU install Alpine Linux Xen LiveCD All Alpine Linux wiki pages
CentOS 5	centos.org	CentOS is an Enterprise-class Linux Distribution derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor. CentOS conforms fully with the upstream vendor's redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork.) CentOS is free.	CentOS 5.x includes the Xen Project 3.4 Hypervisor, and can be used as a dom0 and domU out-of-the-box CentOS / RHEL 6 as Xen Project Host All CentOS wiki pages
CentOS 6	centos.org	CentOS is an Enterprise-class Linux Distribution derived from sources freely provided to the public by a prominent North American Enterprise Linux vendor. CentOS conforms fully with the upstream vendor's redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork.) CentOS is free.	CentOS 6.0 - 6.3 does not include Xen Project software, but you can get support from various sources. The following articles may be useful CentOS / RHEL 6 as Xen Project Host CentOS 6 as Xen Project Guest (32 bit) CentOS 6 as Xen Project Guest (64 bit) CentOS 6 as XCP Guest All CentOS wiki pages CentOS 6.4+ does include Xen Project support and can be used as a dom0 and domU out-of-the-box, thanks to the Xen4CentOS project Xen4CentOS Xen4CentOS6 Release Notes Xen4CentOS6 Quickstart Guide All CentOS wiki pages Xen packages in CentOS 6 and commercial support are also available from "Xen made easy!" Xen packages from "Xen made easy!"
Debian	debian.org	The Debian project produces an entirely free operating system that empowers its users to be in control of the software running their computers.	Debian as Xen Host Debian Guest Installation Using Debian Installer XCP toolstack on a Debian-based distribution All Debian wiki pages
Fedora	fedoraproject.org	Fedora is a RPM-based distribution with a 6-month release cycle, and is the community-supported base of RHEL releases.	Fedora 16 and newer provides full Xen support out of the box. Fedora Host Installation Fedora Test Days All Fedora wiki pages
FreeBSD	freebsd.org	FreeBSD® is an advanced operating system for modern server, desktop, and embedded computer platforms.	How to Install a FreeBSD domU on a Linux Host All FreeBSD Wiki Pages
Finnix	finnix.org	Finnix is a sysadmin utility Linux LiveCD, and includes out-of-the-box Xen Project guest support.	Finnix for VPS providers All Finnix wiki pages
Gentoo Linux	gentoo.org	Gentoo Linux is a special flavor of Linux that can be automatically optimized and customized for just about any application or need. Extreme performance, configurability and a top-notch user and developer community are all hallmarks of the Gentoo experience.	Gentoo Xen page All Gentoo wiki pages
NetBSD	netbsd.org	NetBSD is a free, fast, secure, and highly portable Unix-like open source operating system.	NetBSD Xen Overview NetBSD Xen HowTo Compiling Xen From Source on NetBSD All NetBSD wiki pages
Oracle Linux	oracle.com	Oracle Corporation distributes Oracle Linux the Unbreakable Enterprise Kernel. Oracle states that the Unbreakable Enterprise Kernel is compatible with RHEL, Oracle middleware and 3rd-party RHEL-certified applications. Oracle Linux supports KVM, Xen Project, and Oracle VM Server for x86, which is based on Xen.	All Oracle Linux wiki pages
openSuSE	opensuse.org	openSuSE is a free and Linux-based operating system for your PC, Laptop or Server.	All OpenSuSE wiki pages
Red Hat Enterprise Linux (RHEL) 5.x	redhat.com	RHEL 5.x includes the Xen Project 3.4 Hypervisor as well as a Xen Project-enabled kernel, and can be used as a dom0 and domU	CentOS / RHEL 6 as Xen Project Host All RHEL wiki pages
Red Hat Enterprise Linux (RHEL) 6.x	redhat.com	RHEL 6.x does not include the Xen Project Hypervisor. But, a Dom0 capable kernel, Xen Project hypervisor, and libvirt packages for use with RedHat Enterprise Linux 6 and its derivatives are available from either the Xen4CentOS project or the "Xen made easy!" effort.	Xen4CentOS Xen Project packages from "Xen made easy!"
Ubuntu	ubuntu.com	Fast, secure and stylishly simple, the Ubuntu operating system is used by 20 million people worldwide every day.	Ubuntu Xen Project Wiki page (includes both host and guest installation) Virtualization Docs XCP Ubuntu Install All Ubuntu wiki pages

Getting Help!

The Xen community are helpful and friendly people. We are here for you. There are several ways to get help and keep on top of what is going on!

• Read News!
• Read Documentation!
• Contact other users, to ask the questions and discuss Xen, XCP or other Xen related projects

News Sources

• Xen Home page aggregates selected news stories and blog posts
• The Xen Blog covers technical and community related stories
• xen-announce mailing list is a low frequency mailing list for important announcements

Documentation

Documentation for projects hosted on Xen.org is available on the Xen Wiki. Our wiki is active and community maintained. It contains a lot of useful information and uses categories extensively to make it easy to find information. You may also want to check:

• Category:Manual
• Category:HowTo
• Category:FAQ
• For more applicable categories, see the front page

Mailing Lists

Search Mailing Lists All Xen.org mailing lists are archived using the MarkMail system at xen.markmail.org. Before you ask a question, it is worth checking whether somebody else has asked the question before

Main Mailing Lists Xen.org maintains a number of mailing lists for users of Xen, the Xen Cloud Platform and other projects. English is used by readers on this list.

• xen-users is the list for technical support and discussions for the Xen hypervisor. If you are not sure where your question belongs start here!
• xen-api is the list for technical support and discussions for the Xen Cloud Platform (XCP).
• xen-arm is the list for technical support and discussions for the Xen ARM project.

IRC

Internet Relay Chat (IRC) is a great way to connect with Xen community members in real time chat and for support.

• ##xen is the channel for technical support and discussions for the Xen hypervisor. If you are not sure where your question belongs start here!
• Check out our IRC page if you are not familiar with IRC.

Other places

There are a number of other places, where you can get help on Xen and XCP. For example:

• Xen Questions on serverfault
• XCP & XenServer Questions on serverfault
• Many Linux Distributions that support Xen have a dedicated list - see the table in this link

Raising Bugs

If you find a bug, you can report bugs against Xen, or the Xen Cloud Platform. Before you raise a bug, please read Reporting Bugs!

Roadmaps, Release Cadence, Maintenance Releases

The xen.org community aims to release the Xen Hypervisor on an approximate cadence of 9 months. Roadmap information is tracked at Xen Roadmap. You can find information on the maintenance releas cycle at Xen Maintenance Releases.

Also See

Installation

• Getting Started guides a new user through key decisions to be made
• Category:Host Install contains guides on how to install a control domain
• Category:Guest Install contains guides on how to install a variety of guests
• Category:Host Configuration contains documents related to bootloader, console and network configuration
• Guest VM Images provides pointers to various preinstalled guest images.
• LiveCD provides pointers to Live CDs for Xen

Release Information

• Category:Manual contains Xen manual documents
• Category:Release Notes contain Xen release notes
• Xen Release Features contains a matrix of features against Xen versions
• Category:Xen 4.2 contain articles related to Xen 4.2 features, benchmarks, planning, etc.

Specialist Topics: Networking, Performance, Security, NUMA, VGA, ...

Specialist topics on Xen:

• Category:Networking contains articles related to networking
• Category:NUMA contains all articles related to the running (or to improving the support for doing so) of the Xen Hypervisor on NUMA architectures
• Category:Performance contains documents, tuning instructions and benchmarks related to the performance of Xen, XCP, etc.
• Category:Security contains documents related to Xen, XCP, etc. security
• Category:VGA contains documents related to VGA, VTd, GPY passthrough, etc.

FAQs, HowTos, ...

• Category:FAQ contains Xen FAQs
• Category:HowTo contains various HowTo's
• Category:Tutorial contains various Tutorials