Mainline Linux Kernel Configs

Getting the Current Stable Version

The current kernel tarballs are linked from the front page of kernel.org. For stable use it is recommended that you use the latest "stable" point release. Alternatively you can clone from the stable git tree at git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git (gitweb).

Getting the Current Development Version

There are two main development trees which are interesting to Xen users/developers.

Mainline Kernel

The first is Linus Torvald's mainline git tree which can be found at git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (gitweb).

This tree has only one active branch, master, but also contains tags for each release and release candidate which can be useful for providing a more concrete baseline for a test or bug report than whatever the current tip happens to be.

Xen Development Tree

The second is Konrad Rzeszutek Wilk's (the Xen Linux maintainer) tree which is where most day-to-day Xen development work ends up en route to Linus' tree. Konrad's tree can be found at git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git (gitweb).

The repository has numerous topic branches to track individual lines of development, and a couple of roll-up branches which contain everything merged together for easy compilation and running.

NOTE! All active git branches require at least Xen 4.0.1, using older version (4.0.0 or older) will cause problems, xend not starting, etc.

Current active branches are:

• linux-next - patches that are queued up to go to Linus Torvald's tree. This includes bug-fixes, new features, etc.
• testing (in Konrad's repository) is off patches being run/developed and obviously going through testing - which means they might not compile or work properly.

Using git

The primary version control system used for Linux kernel development is git. There are numerous git HOWTOs on the web, we

To clone Linus' tree:

   $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux
   $ cd linux

To clone Konrad's tree:

   $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git linux
   $ cd linux

This will automatically check out the 'origin/master' default branch.

If you are using Konrad's tree then you most probably want to use the linux-next (or upstream/xen) branch:

   $ git reset --hard
   $ git checkout -b xen-next origin/linux-next
   Branch xen-next set up to track remote branch refs/remotes/origin/linux-next
   Switched to a new branch "xen-next"
   $ git pull
   $ git log | less

Later when you want to update the tree use:

   $ git pull

Configuring the Kernel

Configure as normal; you can start with your current .config file or use make defconfig and then:

   $ make menuconfig

NOTE0: Make sure you have correct CPU type (Processor Family) set in the kernel configuration, Xen Dom0 options won't show up at all if you have too old CPU selected (too old means a CPU that doesn't support PAE; Pentium Pro was the first CPU to have PAE).

NOTE1: If you're building 32 bit version of the kernel, you first need to enable PAE support, since Xen only supports 32 bit PAE kernels nowadays. Xen kernel build options won't show up at all before you've enabled PAE for 32 bit builds (Processor type and features -> High Memory Support (64GB) -> PAE (Physical Address Extension) Support). PAE is not needed for 64 bit kernels.

Configuring the Kernel for domU Support

1. If building 32 bit kernel make sure you have CONFIG_X86_PAE enabled (which is set by selecting CONFIG_HIGHMEM64G)
   • non-PAE mode doesn't work in 2.6.25, and has been dropped altogether from 2.6.26 and newer kernel versions.
2. Enable these core options (Processor type and features| Paravirtualized guest support]
   • CONFIG_PARAVIRT=y
   • CONFIG_XEN=y
   • CONFIG_PARAVIRT_GUEST=y
   • CONFIG_PARAVIRT_SPINLOCKS=y
3. And Xen pv console device support (Device Drivers|Character devices
   • CONFIG_HVC_DRIVER=y
   • CONFIG_HVC_XEN=y
4. And Xen disk and network support (Device Drivers|Block devices and Device Drivers|Network device support)
   • CONFIG_XEN_FBDEV_FRONTEND=y
   • CONFIG_XEN_BLKDEV_FRONTEND=y
   • CONFIG_XEN_NETDEV_FRONTEND=y
5. And the rest (Device Drivers|Xen driver support)
   • CONFIG_XEN_PCIDEV_FRONTEND=y
   • CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
   • CONFIG_XEN_FBDEV_FRONTEND=y
   • CONFIG_XEN_XENBUS_FRONTEND=y
   • CONFIG_XEN_SAVE_RESTORE=y
   • CONFIG_XEN_GRANT_DEV_ALLOC=m
6. And for tmem support:
   • CONFIG_XEN_TMEM=y
   • CONFIG_CLEANCACHE=y
   • CONFIG_FRONTSWAP=y
   • CONFIG_XEN_SELFBALLOONING=y

Configuring the Kernel for dom0 Support

NOTE: Xen dom0 support depends on ACPI support. Make sure you enable ACPI support or you won't see Dom0 options at all.

In addition to the config options above you also need to enable:

• CONFIG_X86_IO_APIC=y
• CONFIG_ACPI=y
• CONFIG_ACPI_PROCFS=y (optional)
• CONFIG_XEN_DOM0=y
• CONFIG_PCI_XEN=y
• CONFIG_XEN_DEV_EVTCHN=y
• CONFIG_XENFS=y
• CONFIG_XEN_COMPAT_XENFS=y
• CONFIG_XEN_SYS_HYPERVISOR=y
• CONFIG_XEN_GNTDEV=y
• CONFIG_XEN_BACKEND=y
• CONFIG_XEN_NETDEV_BACKEND=m
• CONFIG_XEN_BLKDEV_BACKEND=m
• CONFIG_XEN_PCIDEV_BACKEND=m
• CONFIG_XEN_PRIVILEGED_GUEST=y
• CONFIG_XEN_BALLOON=y
• CONFIG_XEN_SCRUB_PAGES=y

If you're using RHEL5 or CentOS5 as a dom0 (ie. you have old udev version), make sure you enable the following options as well:

• CONFIG_SYSFS_DEPRECATED=y
• CONFIG_SYSFS_DEPRECATED_V2=y

For more current Xen related config options check the example .config files from the troubleshooting section, and check the 2.6.18-to-2.6.31-and-higher wiki page.

Building the kernel

After having downloaded and configured the kernel source as above building the kernel is as simple as:

   $ make all -j <2*number_of_cores>

The -j option tells make how many threads to use, a good number is something like twice the number of processor cores in your system since this allows for I/O (e.g. reading the code from disk) to be interleaved with processing (e.g. compiling the code).

Installing the Kernel

The foo-pkg Targets

The kernel contains several makefile targets which can create packages in various formats. These packages include a baseline level of integration with the relevant distros (although not likely to be as good as the distros own packages they are good enough). Even if the distro integration is lacking simply being able to cleanly uninstall the package is a big advantage.

The full list of package targets can be found in the output of make help:

   $ make help
   [...]
   Kernel packaging:
     rpm-pkg             - Build both source and binary RPM kernel packages
     binrpm-pkg          - Build only the binary kernel package
     deb-pkg             - Build the kernel as a deb package
     tar-pkg             - Build the kernel as an uncompressed tarball
     targz-pkg           - Build the kernel as a gzip compressed tarball
     tarbz2-pkg          - Build the kernel as a bzip2 compressed tarball
     tarxz-pkg           - Build the kernel as a xz compressed tarball
     perf-tar-src-pkg    - Build perf-3.8.0-rc2.tar source tarball
     perf-targz-src-pkg  - Build perf-3.8.0-rc2.tar.gz source tarball
     perf-tarbz2-src-pkg - Build perf-3.8.0-rc2.tar.bz2 source tarball
     perf-tarxz-src-pkg  - Build perf-3.8.0-rc2.tar.xz source tarball

The most interesting targets are the rpm-pkg, binrpm-pkg and deb-pkg targets which cover a wide variety of distros. To use simply invoke with make:

   $ make deb-pkg

Once this command has completed you should find a suitable package in the parent directory which can be installed with rpm or dpkg in the usual way, e.g.

   $ sudo dpkg -i ../linux-image-<version>_<version>_<arch>.deb

Manual Installation

If you want to install the kernel manually then you can simply do, as root:

   # make modules_install
   # make install

This will install the necessary files to /boot and /lib/modules with no tracking (for this reason it is highly advised to use the foo-pkg targets).

Once you have installed you may need to rebuild the initramfs by hand (some distros do this automatically as part of the above steps). The tool to do this is usually Distro dependent, e.g. dracut (recent Fedora & Red Hat) mkinitrd (Older Fedora & Red Hat) or mkinitramfs (Debian & Ubuntu). e.g.:

   # mkinitrd -f /boot/initrd-<version>.img <version>
   # mkinitramfs <version>

Obsolete Debian/Ubuntu make-kpkg Method

Older versions of Debian/Ubuntu used a tool called `make-kpkg` to both build and package the kernel. However this tool is now obsolete. It is recommended to use the above methods instead.

   $ sudo make -j <number_of_cores +1>
   $ sudo apt-get install kernel-package fakeroot
   $ sudo CONCURRENCY_LEVEL=<number_of_cores +1> make-kpkg clean
   $ sudo CONCURRENCY_LEVEL=<number_of_cores +1> fakeroot make-kpkg --initrd --append-to-version=-pv kernel-image kernel-headers
   $ sudo dpkg -i ../<linux-image-3.0.XXX.deb>

The first line will make the kernel with the hidden configuration file (your file hopefully). You might have to do this several times because you will encounter errors that will require turning modules off, and start compiling again. It is a good practice to use make clean when you change many modules at a time. The next line is used for dpkg but is not really that important. the sudo fakeroot line will create your ramdisk needed to boot. This is the Debian way of making the image usable this will event copy it to boot but will not update grub.

Update modules

You may need to update some new modules that your newly created kernel requires with Xen. Under new enough versions of Xen (e.g. 4.2 and later) the Xen initscripts should take care of this for you.

For older Xen versions you may need to do this yourself by adding the modules to the relevant configuration file. Under Debian and Ubuntu style systems this is /etc/modules.

The following modules may be required:

   xen-evtchn
   xen-gntdev
   xen-netback
   xen-blkback
   xenfs
   blktap