Archive/Making XCP easy to build

Making XCP components easy to build

This page describes a proposal for making XCP easy to build. Please send feedback to the list: xen-api@lists.xen.org

Problem overview

Why is this important?

Before you can propose a change to an open-source project you need to be able to build a binary and test it. We want building an XCP component to be as easy and as cheap as possible, to allow any interested person to contribute.

Where are we now?

The current XCP build system is sophisticated and tightly-coupled to Citrix infrastructure. It is possible to manually create 'snapshots' of the build system state at a point in time to help third parties rebuild specific components, but these 'snapshots' are large (and therefore slow to download), quickly become out of date and have to be manually refreshed. The build system targets specific distribution versions using a system of "chroots" which are only needed if you compiling a package for a specific binary XCP version. The build system has a notion of "components" rather than "packages" and the dependencies between components are difficult to manage via Makefiles. The build system emphasises repeatability of whole-CD rebuilds and does not emphasise fast incremental rebuilds of individual packages (should they be possible in theory).

The proposed future state

In future we will focus on

1. producing high-quality, easy-to-build packages for a number of common distributions including CentOS, Debian, Ubuntu.
2. improving and simplifying the *developer user experience* so it is quicker and easier to develop for XCP.

User example 1

Alice manages a fleet of virtualisation hosts running Xen. A security vulnerability is discovered and a patch is quickly created to fix the problem. Alice fetches the source for the stable version she's currently running:

sudo yumdownloader --source xenopsd

She is then able to backport the patch, recompile the RPM and roll out the update to all her hosts in the usual way.

User example 2

Bob is building a new virtualisation feature where a VM may be transparently migrated between Xen and KVM hosts. The feature requires simultaneous changes to several components at once. Bob uses a simple source-based package management tool to install the bleeding-edge development versions of all the relevant components and their dependencies on his development box e.g.

source-package-mgr install xapi

He can then edit each component and build it in the usual way i.e.

./configure && make && make test && make install

After making changes to a base component (e.g. a xen interface), he can perform an incremental rebuild of a high-level component as follows:

source-package-mgr reinstall xapi

When Bob is satisfied that his changes work, he can trigger the build of a yum repo containing a consistent set of binary RPMs with his changes in. These can then be submitted to an automated QA system for system-level tests.

User example 3

Charlie reads a techie blog linked from hacker news describing a cool set of virtualisation tools. He first installs the base stable software using

sudo yum install xcp

After playing for a while, he notices a typo in a man page. He has 5 minutes free so she decides she's going to try to fix this himself because it's the right thing to do (TM). Since the manpage belongs to the "xapi" binary he googles for "xapi source code" and quickly finds a link to a repository hosted on github. She:

1. clicks on the "fork" button to clone the repository
2. creates a lightweight feature branch in the standard way
3. fixes the typo
4. uses "./configure && make && make test && make install" to make sure the project still builds (just in case)
5. creates a pull request with the fix

User example 4

Dave runs a test cluster and would like to provide early feedback on the state of an in-development XCP feature. He first installs his distribution of choice (e.g. CentOS, Debian, Ubuntu) and then modifies his package manager to point to a "XCP development" package archive hosted on http://xenbits.xen.org/. He then upgrades his XCP components in the usual way and is able to test the latest build of the new feature in his environment.

"Early adopter" Dave finds and reports some bugs, which are quickly fixed in the development branch. Since the development packages are continuously being rebuilt and published, Dave is able to immediately upgrade his packages and verify the fixes.

Proposed technical details

We shall use 3 kinds of package:

1. stable, production-ready packages in upstream distributions. These will contain the periodic XCP releases. We will strive to make these packages easy to build and maintain, to reduce the burden on package maintainers. The packages will conform to all the required distribution standards.
2. unstable, development binary packages in custom package archives on xenbits.xen.org. These will contain the latest and greatest developer builds, suitable for QA system test and for anyone to try out an in-development feature and provide feedback. These packages will contain automatically-generated metadata and generally will not be expected to conform to upstream distribution standards.
3. distro-agnostic source packages: these are aimed at developers, particularly to help make cross-component changes. The packages should support quick, accurate, ideally-incremental rebuilds to reduce the barrier to entry for a new developer and to reduce the develop/test cycle time.

Implementation Phases

The requirements have been broken down into phases, so we can get the important stuff done as soon as possible/

Phase 1: the absolute minimum

Once this phase is complete it should be possible to

1. rebuild RPMs containing custom fixes (see user example 1)
2. design, build, test and contribute a fix to a XCP component (see user example 3)

We need:

1. public repositories on github containing both the latest development branches and all future stable versions
2. each repository should contain a README explaining how to build, test and propose fixes
3. a "style guide" on the xen wiki, linked to from the README, which requires that each component
   1. uses the standard build interface:

     "./configure && make && make test && make install"

1. 1. contains the necessary RPM "spec" file to allow SRPM generation and binary rebuids
   2. contains the relevant LICENSE file
   3. contains the relevant Changelog entries
2. redefine a "release" to include a published yum repo

    with all the source and binary RPMs 

Phase 2: smarter tools

Once this phase is complete it should be possible to easily develop new features, even when they require changes to multiple components (see user example 2).

We need a developer-oriented tool which can:

1. understand a concept of a "component" with versions and dependencies on other components.
2. automatically "clone" all the repositories for a component, optionally including their dependencies. This is similar to the Citrix build system's "make component-myclone" step.
3. automatically "mirror" or "update" either all the repositories in XCP or all the repositories needed by a component. This allows offline builds of components. Note this needs to include automatically handling the creation of new repositories.
4. rebuild a component after changes have been made to a dependency (by running "./configure && make && make test && make install" in all the relevant places). This is similar to the Citrix build system's "make component-build" step.

Note this tool is basically a source packaging system, many types of which already exist e.g. BSD ports, homebrew, nix, opam.

Phase 3: the more binary packages the better

Once this phase is complete it should be possible for an early adopter to upgrade XenServers to the latest development snapshots to test fixes (see user example 4).

We need to:

1. extend the developer tool to generate skeleton RPM .spec files from the component dependencies (and similar metadata for any other distributions we wish to support). Note these skeleton .spec files will not be the same as the maintained versions shipped in distributions.
2. extend the developer tool to trigger the build of the SRPMs and binary RPMs via the generated .spec files
3. create another tool which can repeatedly publish the resulting yum repo on xenbits.xen.org.

Phase 4: the more distributions the better

We shall repeat the work from phase 3 with every other distribution we wish to support.