Xen 4.0 Release Notes

Xen 4.0 release notes

Xen 4.0.0 release announcements:

• Xen hypervisor 4.0.0 was released on 07 Apr 2010.
• Official release announcement email from xen-devel mailinglist: http://lists.xensource.com/archives/html/xen-devel/2010-04/msg00268.html
• blog.xen.org announcement: http://blog.xen.org/index.php/2010/04/13/xen-org-announces-xen-4-0-availability/
• Citrix announcement: http://citrix.com/English/NE/news/news.asp?newsID=1864748
• Download url: http://xen.org/download/index_4.0.0.html

Xen 4.0.1 release announcements:

• Xen hypervisor 4.0.1 was released on 25 Aug 2010.
• Official release announcement email from xen-devel mailinglist: http://lists.xensource.com/archives/html/xen-devel/2010-08/msg01526.html
• Download url: http://xen.org/download/index_4.0.1.html

Xen 4.0.2 release announcements:

• Xen hypervisor 4.0.2 was released on 15 Jun 2011.
• Official release announcement email from xen-devel mailinglist: http://lists.xensource.com/archives/html/xen-devel/2011-06/msg01050.html
• Download url: http://xen.org/download/index_4.0.2.html

Xen 4.0.3 release announcements:

• Xen hypervisor 4.0.3 was released on 21 Oct 2011.
• Official release announcement email from xen-devel mailinglist: http://lists.xen.org/archives/html/xen-devel/2011-10/msg01555.html
• Download url: http://xen.org/download/index_4.0.3.html

Xen 4.0.4 release announcements:

• Xen hypervisor 4.0.4 was released on 10 Aug 2012.
• Official release announcement email from xen-devel mailinglist: http://lists.xen.org/archives/html/xen-devel/2012-08/msg00928.html
• Download url: http://xen.org/download/index_4.0.4.html

Xen 4.1.x release announcements:

• Xen4.1 info can be found from Xen4.1 wiki page.

For a full list of available Xen hypervisor downloads see: http://xen.org/products/xen_archives.html .

Overview of Xen 4.0

Xen 4.0 is the core hypervisor with basic command line management tools. You can use it with your favourite Linux distribution to build your own secure, high performance virtualization solution. Some users add additional third party management tools and interfaces to create their own virtualization platform. More information about the available external 3rd party management tools and (web) interfaces in the XenManagementTools wiki page.

If you don't want to build your own solution from multiple components and you're looking for a full platform, which is more complete, integrated and tested as a whole, please check out the Xen Cloud Platform (XCP): http://www.xen.org/products/cloudxen.html

Xen 4.0 new features

See the end of this wiki page for more links/documentation.

• Better performance and scalability: 128 vcpus per guest, 1 TB of RAM per host, up to 1 TB of RAM per HVM guest or 512 GB of RAM per PV guest, 128 physical CPUs per host (as a default, can be compile-time increased to lots more).
• blktap2 for VHD image support, including high-performance snapshots and cloning. Wiki link: blktap2
• Improved IOMMU PCI passthru using hardware accelerated IO virtualization techiques (Intel VT-d and AMD IOMMU). Wiki links: XenPCIpassthrough and VTdHowTo
• VGA primary graphics card GPU passthru support to an HVM guest for high performance 3D graphics and hardware accelerated video using direct access to the graphics card GPU from the guest OS. Wiki links: XenVGAPassthrough
• TMEM allows improved utilization of unused (for example page cache) PV guest memory. more information: http://oss.oracle.com/projects/tmem/
• Memory page sharing and page-to-disc for HVM guests: Copy-on-Write sharing of identical memory pages between VMs.This is an initial implementation (tech preview) and will be improved in upcoming releases.
• New Linux pvops dom0 kernel 2.6.32.x as a default. You can also use linux-2.6.18-xen dom0 kernel with Xen 4.0 hypervisor if you want. Wiki link: XenDom0Kernels
• Netchannel2 for improved networking acceleration features and performance, smart NICs, multi-queue support and SR-IOV functionality.
• Online resize of guest disks without reboot/shutdown.
• Remus Fault Tolerance: Live transactional synchronization of VM state between physical servers. run guests synchronized on multiple hosts simultaneously for preventing downtime from hardware failures. Remus support in Xen 4.0.0 requires linux-2.6.18-xen dom0 kernel. Pvops dom0 kernel (2.6.32) support for Remus is available in Xen 4.0.1. For more information see Remus wiki page.
• RAS features: physical cpu/memory hotplug.
• Libxenlight (libxl): a new C library providing higher-level control of Xen that can be shared between various Xen management toolstacks.
• PV-USB: Paravirtual high-performance USB passthru to both PV and HVM guests, supporting USB 2.0 devices. Wiki link: XenUSBPassthrough
• gdbsx: debugger to debug ELF guests
• Support for Citrix WHQL-certified Windows PV drivers, included in XCP (Xen Cloud Platform). Xen Cloud Platform: http://www.xen.org/products/cloudxen.html
• Pygrub improvements: Support for PV guests using GRUB2, Support for guest /boot on ext4 filesystem, Support for bzip2- and lzma-compressed PV guest bzImage kernels
• And more..

Xen 4.0.1 changes

• Many bugfixes. Upgrading is recommended for all Xen 4.0.0 users.
• Default pvops kernel is now Linux 2.6.32.x.
• Many additions to "xl" and "libxenlight" functionality.
• Pygrub support for booting Xen PV guests using GRUB2 config files (Ubuntu 10.04 LTS).
• Remus Fault Tolerance (FT) support for pvops dom0 kernels.

Xen 4.0.2 changes

• Many bugfixes. Upgrading is recommended for all Xen 4.0.1 users.
• Pygrub support for booting Xen PV domUs using newest version of GRUB2 config files (Debian 6.0 Squeeze).

Xen 4.0.3 changes

• Many bugfixes. Upgrading is recommended for all Xen 4.0.2 users.

Xen 4.0.4 changes

• Many bugfixes and security fixes. Upgrading is recommended for all users of previous Xen 4.0.x versions.

Xen 4.0 hypervisor and supported dom0 Linux kernels

Xen 4.0.1 (and newer 4.0.x versions) includes and builds the new pvops dom0 Linux 2.6.32.x kernel as a default. Xen 4.0.0 used Linux 2.6.31.x pvops kernel as a default. You can also use the old-style linux-2.6.18-xen as the dom0 kernel, or any of the various forward-ports of the 2.6.18 xen patches to newer kernels. Please see the XenDom0Kernels and XenKernelFeatures wiki pages for more information.

Requirements for running Xen 4.0

Xen hypervisor 4.0 supports 32bit PAE-capable CPUs and 64bit x86_64 CPUs. Both Intel and AMD CPUs are supported. Also Intel IA-64 (Itanium) CPUs are supported.

CPU hardware virtualization extensions (Intel VT, AMD-V) are required for running Xen HVM guests (=Windows). Xen PCI passhtru to HVM guests requires hardware IOMMU functionality (Intel VT-d, AMD IOMMU) and support from CPU, BIOS and motherboard chipset.

Xen PV (paravirtualized) guests (Linux, Solaris, BSD, Netware) are supported without CPU/hardware virtualization extensions, so Xen can be used also on old hardware. Xen 32bit PV guests need to be PAE.

Xen 4.0 hypervisor has been tested the most on 64bit CPUs, so it's recommended that you build and run 64bit Xen 4.0 hypervisor. Note that dom0 kernel can still be 32bit PAE even when the hypervisor is 64bit (if you're not able to run full 64bit environment for both Xen+dom0). Xen guests (VMs) can be 32bit when using 64bit Xen hypervisor.

Requirements for building Xen 4.0 from source

New software packages required to compile the Xen hypervisor/tools from source:

• iasl
• xz libs and headers to build pygrub with support for lzma compressed domU kernels.

Note to developers: Xen 4.0 can be built on RHEL5 / CentOS 5, but if you want to modify the sources before building, then you need to have a newer version of "flex" than what's included in the EL5 distributions! For more information please see http://lists.xensource.com/archives/html/xen-devel/2010-03/msg00180.html and http://xenbits.xensource.com/xen-4.0-testing.hg?rev/3138b3885999

Packages required for building from source

Packages required to build Xen 4.0 from source on Fedora 11, 12, 13, 14 or 15, in addition to all the usual development tools and libraries:

yum groupinstall "Development Libraries"
yum groupinstall "Development Tools"
yum install transfig wget texi2html libaio-devel dev86 glibc-devel e2fsprogs-devel gitk mkinitrd iasl xz-devel bzip2-devel pciutils-libs pciutils-devel SDL-devel libX11-devel gtk2-devel bridge-utils PyXML qemu-common qemu-img mercurial libidn-devel
yum install glibc-devel.i686

Required packages for Debian Lenny 5.0 & Squeeze 6.0 / Ubuntu 8.x & 9.x & 10.04:

apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml libx11-dev bison flex

If you have a 64bit version of Debian/Ubuntu you also need this additional package:

apt-get install gcc-multilib 

Also note that in Debian/Ubuntu you'll need "xz-utils" (the new version of lzma library) to be able to build pygrub with support for lzma compressed kernels!:

apt-get install xz-utils

Installation from source

Xen 4.0 source code mercurial (hg) repository and changelog: http://xenbits.xen.org/xen-4.0-testing.hg

You can get Xen 4.0 with mercurial using the following command:

hg clone http://xenbits.xen.org/xen-4.0-testing.hg

Note that this will give you the LATEST version in xen-4.0-testing.hg repository! It might be a release candidate for next release or just some developer snapshot. See the changelog above for current status. You can also get a specific 'tag' from the repository like this:

hg clone -r RELEASE-4.0.2 http://xenbits.xen.org/xen-4.0-testing.hg

Alternatively you can download the Xen 4.0.x source tarball from http://www.xen.org/products/xen_source.html . Note that if you're planning to use Xen 4.0.0 tarball please see the Known Issues below for a note about supported pvops dom0 kernel versions.

NOTE!! during the Xen build process outbound git-protocol needs to be open in your network firewall! Xen build scripts need to access git:// repositories on git.kernel.org and xenbits.xen.org. Some people have reported issues accessing the remote repositories - these problems are most often caused by a broken Internet connection, broken firewall, or a broken NAT router you use.

This example will only build the new Xen 4.0 hypervisor and tools. This method does NOT download or build a (dom0) kernel from the xen.git repository. "make tools" part will download the xen qemu-dm ioemu for hvm guests from xenbits git repository and tools tarballs (unless you've already downloaded them yourself before building).

cd xen-4.0-testing.hg
make xen
make tools
make stubdom

When the compilation process finishes you'll have the Xen hypervisor/tools binaries in "dist/" directory ready for installation:

make install-xen
make install-tools
make install-stubdom

Note some distros (Debian/Ubuntu) might need empty PYTHON_PREFIX_ARG= to install the python libs to correct directory. If you get python errors about missing xen modules, use the following to install xen:

make install-tools PYTHON_PREFIX_ARG=

And then compile your chosen new dom0 kernel, or use your existing dom0 kernel. Also update grub.conf to use the newly installed Xen 4.0 hypervisor.

See XenDom0Kernels wiki page for information about all the available Xen Dom0 capable kernels, and XenParavirtOps wiki page for more detailed information about Xen pvops dom0 kernels.

You can also compile Xen 4.0 together with the new default pvops dom0 kernel (Linux 2.6.32.x in Xen 4.0.1 and newer versions). This method downloads the kernel sources from xen.git repository during the build process:

make world <- Do not use this if you use make menuconfig sort of step

This will download the latest pvops dom0 kernel 2.6.32.x from Jeremy's xen.git repository during the build process. See the XenParavirtOps wiki page for more information about different pvops dom0 kernel versions. Also see the Known Issues below for supported pvops dom0 kernel versions.

NOTE about the old Xenlinux linux-2.6.18-xen kernels:

make kernels KERNELS="linux-2.6-xen0 linux-2.6-xenU"

This will download and build the linux-2.6.18-xen based dom0 and domU kernels. You probably want to use the new Linux 2.6.32 pvops kernel instead.

Keeping the Xen 4.0 source tree up-to-date and tracking upstream changes

Later if you want to track the changes made (patches added) to xen-4.0-testing.hg repository do this to update the tree:

cd xen-4.0-testing.hg
make clean
hg pull
hg update
hg log | less
<!-- #verify you see the latest changes in the log <and continue as above to build and install it>

-->

Example grub configuration for Xen 4.0 with a pvops dom0 kernel

Example entry for Xen dom0 /boot/grub/grub.conf:

title        Xen 4.0, Linux kernel 2.6.32.25
root         (hd0,0)
kernel       /xen-4.0.gz dom0_mem=1024M loglvl=all guest_loglvl=all
module       /vmlinuz-2.6.32.25 root=/dev/sda1 ro nomodeset
module       /initrd.img-2.6.32.25

Obviously you need to modify the example grub.conf entry above to have correct values for your setup, especially for the root= parameter. Check the existing grub.conf entries and make the new entry similar to them.

Binary packages of Xen 4.0

There are no binary packages available from xen.org. Some Linux distributions, and some third parties distribute binaries of Xen 4.0.

• RHEL5 / CentOS5 users can grab Xen 4.0 binary rpms from: http://gitco.de/repo/
• Fedora 14 (and F15 rawhide) includes Xen 4.0.1 binary rpms. They're available from all Fedora mirrors as binaries and as src.rpm, for example http://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/14/Everything/source/SRPMS/xen-4.0.1-6.fc14.src.rpm .
• Fedora developer M A Young has "xendom0" kernel binary rpms available to be used together with Xen 4 rpms: repository http://repos.fedorapeople.org/repos/myoung/dom0-kernel/ .
• Fedora 13 users can get Xen 4.0 RPMS + Pvops RPM from: http://www.pdx-pc.de/2010/08/xen-4-0-on-fedora-13/
• Fedora 13 users can get Xen 4.0.1 RPMS + Pvops 2.6.32 from: http://www.pdx-pc.de/downloads/repo/fc13/testing/  !! Consider these as testing !!
• Debian 6.0 ("Squeeze") and unstable (sid) includes Xen 4.0.1 deb packages. Remember in Debian you need to install the separate "xen-qemu-dm-4.0" deb to get Xen HVM support.

Fedora users can grab Xen 4.0.0 src.rpm from http://pasik.reaktio.net/fedora/xen-4.0.0-0.7.fc12.src.rpm and Xen 4.0.1 src.rpm from http://pasik.reaktio.net/fedora/xen-4.0.1-6.fc14.src.rpm . You can then use "rpmbuild -bb" to build binary rpm packages on your system. These src.rpm packages can be rebuilt on Fedora 13. For more information about Fedora rpms please see http://fedoraproject.org/wiki/Features/XenPvopsDom0 .

Tutorials to install Xen 4.0

• Xen 4.0 on Ubuntu 10.04: http://bderzhavets.wordpress.com/2010/04/24/set-up-ubuntu-10-04-server-pv-domu-at-xen-4-0-dom0-pvops-2-6-32-10-kernel-dom0-on-top-of-ubuntu-10-04-server/
• Xen 4.0 + virt-manager on Ubuntu 10.04: http://bderzhavets.wordpress.com/2010/06/02/setup-libvirt-0-8-0-xen-4-0-on-top-of-ubuntu-10-04-server-via-daniel-baumann-virtualization-ppa/
• Xen 4.0 on Fedora 13: http://bderzhavets.wordpress.com/2010/05/18/set-up-libvirt-0-7-7xen-4-0-on-top-fedora-13-rc2/
• Xen 4.0 on Fedora 12 with Xenlinux dom0: http://pkill.info/b/2256/setting-up-stable-xen-domu-with-fedora-unmodified-fedora-12-on-top-of-xenified-fedora-12-dom0-with-xen-4-0-0/
• Xen 4.0.1 on Fedora 13 (step-by-step tutorial): Fedora13Xen4Tutorial
• CentOS 5.5 Xen 4.0 tutorial: http://ppadala.net/blog/2010/09/how-to-setup-xen4-pvops-2-6-32-x-on-centos-5-5/
• CentOS 5.5 Xen 4.0 tutorial using gitco rpms: http://ppadala.net/blog/2010/09/how-to-setup-xen4-and-pvops-2-6-32-x-on-centos-5-5-using-gitco-repository/
• Redhat Enterprise Linux 6 (RHEL6) with Xen 4.0: RHEL6Xen4Tutorial .

Known issues

• pvgrub is known to have a couple of issues in 4.0.0 release. This might prevent the guest using pvgrub from starting. Patches to fix the bugs can be found from xen-devel mailinglist archives, and they are included in Xen 4.0.1 release.
• guest won't start if the disk image path has spaces in the file or directory names. You'll get "Error: Device 2049 (vbd) could not be connected." and "/etc/xen/scripts/block failed; error detected." messages. This also happens if the real filename behind a symlink has spaces in the file or directory name.
• Latest pvops dom0 kernel versions (both xen/stable-2.6.31.x and xen/stable-2.6.32.x) after June 2010 require at least Xen 4.0.1-rc3 or newer to work properly! Using Xen 4.0.0 with these kernels causes xenstored and xend fail to start. See XenParavirtOps for more information about the issue. Note that if you download the Xen 4.0.0 tarball and build the default pvops dom0 kernel with "make world", you'll get the latest kernel version, which won't work with the 4.0.0 release! It's recommended to use Xen 4.0.1 or the latest xen-4.0-testing.hg version, because the latest pvops dom0 kernels do work out-of-the-box with these hypervisors.
• Xen 4.0.1 uses "tap2:" prefix for blktap2 disks in the "/etc/xen/<guest>" configuration files. Please see blktap2 wiki page for more information about the new syntax.
• In Xen 4.0.1 there's a bug in HVM guest save/restore/livemigration on Intel CPUs, related to XSAVE feature. As a workaround you can add "no-xsave" as an option to Xen command line in grub.conf. This issue is fixed in Xen 4.0.2-rc1-pre with this patch: http://xenbits.xen.org/xen-4.0-testing.hg?rev/16867267ac12 .
• Some Xen 4.0 users have had problems with interrupts getting frozen on Intel Xeon platforms, causing raid adapters to freeze and disk IO to stall. This is due to hpet broadcast (hpet timer migration) issues. The quick fix is to add "cpuidle=off" or "max_cstate=1" cmdline parameters for xen.gz in grub.conf. See these emails for more information: http://lists.xensource.com/archives/html/xen-devel/2010-09/msg00556.html and http://lists.xensource.com/archives/html/xen-devel/2010-09/msg01749.html .

More information about new features and general tips

• PV-USB: http://wiki.xensource.com/xenwiki/XenUSBPassthrough
• PCI passthru: http://wiki.xensource.com/xenwiki/XenPCIpassthrough
• VGA/Graphics card passthru: http://wiki.xensource.com/xenwiki/XenVGAPassthrough
• New pvops dom0 Linux kernel: http://wiki.xensource.com/xenwiki/XenParavirtOps
• Pvops dom0 kernel DRM/KMS/TTM graphics issues: http://wiki.xensource.com/xenwiki/XenPVOPSDRM
• All available dom0 Linux kernels: http://wiki.xensource.com/xenwiki/XenDom0Kernels
• Features in different Xen kernels: XenKernelFeatures
• Xen PVSCSI passthru: http://wiki.xensource.com/xenwiki/XenPVSCSI
• Xen blktap2: blktap2
• Fault Tolerance for Xen, Transparent High Availability for VMs: Remus
• Xen Best Practices: http://wiki.xensource.com/xenwiki/XenBestPractices
• Help with various common issues/problems: http://wiki.xensource.com/xenwiki/XenCommonProblems
• Xen wiki main page: http://wiki.xensource.com/xenwiki
• Help and tips for migrating from linux-2.6.18 to pvops dom0 kernel: Migrate_from_Linux_2.6.18_to_2.6.31_and_higher
• List of available 3rd party Xen and XCP management tools and (web) interfaces: XenManagementTools
• Xen PV-on-HVM optimized drivers for HVM (Fully virtualized) Linux guests: XenLinuxPVonHVMdrivers
• Xen HVM guest page sharing patches posted to xen-devel: http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
• Xen HVM guest page sharing discussion/usage: http://lists.xensource.com/archives/html/xen-devel/2010-04/msg00373.html

Links

Xen4.1 features and changes can be found from Xen4.1 wiki page.

How to get Xen running on your machine

   To be able to use Xen on your OS you need to 

• Compile Xen
  
• Compile a Dom0 Kernel which can work with Xen.

The kernel which you see in /boot when you install Linux from a CD will not run with Xen.After you finish above 2 steps you will need to change your grub accordingly.  But both of the steps are not needed if your Linux distributer distributes them as a binary  or rpm or deb package.Ubuntu has Xen related packages until 9.04 and from there on they have not had any Xen specific packages.

So that means if you are using older than Ubuntu 9.04 then it is possible that you use some thing like apt-get install for xen and a suitable Dom0 kernel also from their repositories which can be simply installed via apt-get.Debian some old versions have Xen as a binary package and also a Dom0 kernel so you do not need to compile Xen or a Dom0 kernel either if you plan to use some old release of Debian. This page will help you only if you want to have latest Xen-4.0 kernel and compile things from scratch.If you do not want to use from scratch then XCP is a good choice for you.