This page lists various Xen related development projects that can be picked up by anyone! If you're interesting in hacking Xen this is the place to start! Ready for the challenge?
To work on a project:
- Find a project that looks interesting (or a bug if you want to start with something simple)
- Send an email to xen-devel mailinglist and let us know you started working on a specific project.
- Post your ideas, questions, RFCs to xen-devel sooner than later so you can get comments and feedback.
- Send patches to xen-devel early for review so you can get feedback and be sure you're going into correct direction.
- Your work should be based on xen-unstable development tree, if it's Xen and/or tools related. After your patch has been merged to xen-unstable it can be backported to stable branches (Xen 4.2, Xen 4.1, etc).
- Your kernel related patches should be based on upstream kernel.org Linux git tree (latest version).
xen-devel mailinglist subscription and archives: http://lists.xensource.com/mailman/listinfo/xen-devel
Before to submit patches, please look at Submitting Xen Patches wiki page.
If you have new ideas, suggestions or development plans let us know and we'll update this list!
List of projects
Domain support
Upstreaming Xen PVSCSI drivers to mainline Linux kernel
| Date of insert: 01/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: PVSCSI drivers needs to be upstreamed yet. Necessary operations may include:
|
| Outcomes: Not specified, project outcomes |
|
Upstreaming Xen PVUSB drivers to mainline Linux kernel
| Date of insert: 01/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: PVUSB drivers needs to be upstreamed yet. Necessary operations may include:
|
| Outcomes: Not specified, project outcomes |
|
Blkback improvements
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
Description: Blkback requires a number of improvements, some of them being:
- Multiple disks in a guest cause contention in the global pool of pages.
- There is only one ring page and with SSDs nowadays we should make this larger, implementing some multi-page support.
- With multi-page it becomes apparent that the segment size ends up wasting a bit of space on the ring. BSD folks fixed that by negotiating a new parameter to utilize the full size of the ring.
- Add DIF/DIX support [1]
- Further perf evaluation needs to be done to see how it behaves under high load.
|
| Outcomes: Not specified, project outcomes |
|
Netback overhaul
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Wei Liu posted RFC patches that make the driver be multi-page, multi-event channel and with a page-pool. However not all the issues have been addressed yet, meaning that the patches need to be finished and cleaned up yet. Additively, a zero-copy implementation can be considered. Patch serie and discussions:
|
| Outcomes: Not specified, project outcomes |
|
PAT writecombine fixup
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: The writecombine feature (especially for graphic adapters) is turned off due to stability reasons. More specifically, the code involved in page transition from WC to WB gets confused about the PSE bit state in the page, resulting in a set of repeated warnings.
For more informations please check:
|
| Outcomes: Not specified, project outcomes |
|
ACPI S3-state investigation and fixup
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: During Linux-3.3 release the the S3-state was supposed to work including these patches:
But now it is not working anymore. Scope of the project is understanding the reasons for the issues and fix them. |
| Outcomes: Not specified, project outcomes |
|
Parallel xenwatch
| Date of insert: 01/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Xenwatch is locked with a coarse lock. For a huge number of guests this represents a scalability issue. The need is to rewrite the xenwatch locking in order to support full scalability. |
| Outcomes: Not specified, project outcomes |
|
Hypervisor
Microcode uploader implementation
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Intel is working on early implementation where the microcode blob would be appended to the initrd image. The kernel would scan for the appropiate magic constant and load the microcode very early.
The Xen hypervisor can do this similary. |
| Outcomes: Not specified, project outcomes |
|
SEDF Handling of Blocking/Unblocking
| Date of insert: 08/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Dario Faggioli <dario.faggioli@citrix.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: The SEDF scheduler, within Xen, currently deals with events such as a vcpu blocking (in general, stopping running) and unblocking (in general, restarting running) by trying (but failing!) to special case all the possible situations, resulting in the code being rather complicated, ugly, inefficient and hard to maintain. Unified approaches have been proposed for enabling blocking and unblocking in EDF (the algorithm the scheduler uses), without compromising the temporal isolation it provides to the various tasks/vcpus. More specifically, the technique called Constant BandWidth Server (CBS) could easily be implemented. |
| Outcomes: Not specified, project outcomes |
|
SEDF Multiprocessor Support
| Date of insert: 08/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Dario Faggioli <dario.faggioli@citrix.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: The SEDF scheduler, within Xen, does not properly handle SMP systems, unless specific vcpu pinning is specified by the user. That is a big limitation of the current implementation, especially since EDF (the algorithm the scheduler uses) could be easily extended to work in that situations.
The first thing to do would be turn from using one SEDF runqueue per processor one runqueue per "cluster of processors" (like for instance using one runqueue per-L3, as scredit2 is doing). That would already increase the effectiveness of the scheduler on current hardware a lot. After that, a mechanism for balancing and migrating vcpus among different runqueues can be designed and implemented. |
| Outcomes: Not specified, project outcomes |
|
Integrating NUMA and Tmem
| Date of insert: 08/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Dario Faggioli <dario.faggioli@citrix.com>, Dan Magenheimer <dan.magenheimer_AT_oracle_DOT_com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Trascendent memory (Tmem) as a mechanism for discriminating between frequently and infrequently used data, and thus helping allocating them properly. It would be interesting to investigate and implement all the necessary mechanisms to take advantage of this and improve performances of Tmem enabled guests running on NUMA machines. Some more details here. |
| Outcomes: Not specified, project outcomes |
|
Userspace Tools
Convert PyGrub to C
| Date of insert: 15/11/2012; Verified: Not updated in 2020; GSoC: Yes |
| Technical contact: Roger Pau Monné <roger.pau@citrix.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: With the replacement of xend with xl/libxl, PyGrub is the only remaining Python userspace component of the Xen tools. Since it already depends on a C library (libfsimage), converting it to C code should not be a huge effort. PyGrub Python code is no more than a parser to grub and syslinux configuration files.
Some embedded distros (mainly Alpine Linux) already mentioned it's interest in dropping the Python package as a requirement for a Dom0, this will make a Xen Dom0 much more smaller. |
| Outcomes: Not specified, project outcomes |
|
Refactor Linux hotplug scripts
| Date of insert: 15/11/2012; Verified: Not updated in 2020; GSoC: Yes |
| Technical contact: Roger Pau Monné <roger.pau@citrix.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Current Linux hotplug scripts are all entangled, which makes them really difficult to understand or modify. The reason of hotplug scripts is to give end-users the chance to "easily" support different configuration for Xen devices.
Linux hotplug scripts should be analized, providing a good description of what each hotplug script is doing. After this, scripts should be cleaned, putting common pieces of code in shared files across all scripts. A Coding style should be applied to all of them when the refactoring is finished. |
| Outcomes: Not specified, project outcomes |
|
XL to XCP VM motion
| Date of insert: 15/11/12; Verified: Not updated in 2020; GSoC: Yes |
| Technical contact: Ian Campbell |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Currently xl and xapi (the XCP toolstack) have very different concepts about domain configuration, disk image storage etc. This project is to produce one or more command-line tools which support migrating VMs from one toolstack to another.
One tool should be provided which takes an xl configuration file and a reference to an XCP pool, creates a VM in the pool with a close approximation of the same configuration and streams the configured disk image into a selected SR.
A second tool should be provided which performs the opposite operation, i.e. give a reference to a VM residing in an XCP pool it should produce an XL compatible configuration file and stream the VDIs into a suitable format.
These tools could be reasonably bundled as part of either toolstack and by implication could be written in either C, Ocaml or some other suitable language.
The tool need not operate on a live VM but that could be considered a stretch goal.
An acceptable alternative to the proposed implementation would be to implement a tool which converts between a commonly used VM container format which is supported by XCP (perhaps OVF or similar) and the xl toolstack configuration file and disk image format. |
| Outcomes: Not specified, project outcomes |
|
Performance
Performance tools overhaul
| Date of insert: 02/08/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: Generally, works on the performance tool themselves should be listes separately to the Xen_Profiling:_oprofile_and_perf wiki page. |
| Outcomes: Not specified, project outcomes |
|
Upstream bugs!
VCPU hotplug bug
| Date of insert: Sep 1 2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: VCPU hotplug. [2]
To get this its as easy as having this in your guest config:
vcpus=2
maxvcpus=3
And when you launch the guest to play with 'xm vcpu-set 0 2', xm vcpu-set 0 3'
and see the guest forget about one of the CPUs.
This is what you will see in the guest:
udevd-work[2421]: error opening ATTR{/sys/devices/system/cpu/cpu2/online} for writing: No such file or directory
If you instrument udev and look at the code you will find somebody came up with
a "fix": http://serverfault.com/questions/329329/pv-ops-kernel-ignoring-cpu-hotplug-under-xen-4-domu
But the real fix is what Greg outlines in the URL above. |
| Outcomes: Not specified, project outcomes |
|
RCU timer sent to offline VCPU
| Date of insert: Sep 1 2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
Description: [ 0.073006] WARNING: at /home/konrad/ssd/linux/kernel/rcutree.c:1547 __rcu_process_callbacks+0x42e/0x440()
[ 0.073008] Modules linked in:
[ 0.073010] Pid: 12, comm: migration/2 Not tainted 3.5.0-rc2 #2
[ 0.073011] Call Trace:
[ 0.073017] <IRQ> [<ffffffff810718ea>] warn_slowpath_common+0x7a/0xb0
which I get with this guest config:
vcpus=2
maxvcpus=3
Here is what Paul says: https://lkml.org/lkml/2012/6/19/360 |
| Outcomes: Not specified, project outcomes |
|
CONFIG_NUMA on 32-bit.
| Date of insert: Sep 1 2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: http://www.spinics.net/lists/kernel/msg1350338.html
I came up with a patch for the problem that William found:
http://lists.xen.org/archives/html/xen-devel/2012-05/msg01963.html
and narrowed it down the Linux calling xen_set_pte with a PMD flag
(so trying to setup a 2MB page). Currently the implemenation of xen_set_pte
can't do 2MB but it will gladly accept the argument and the multicall will
fail.
Peter did not like the x86 implemenation so I was thinking to implement
some code in xen_set_pte that will detect that its a PMD flag and do
"something". That something could be either probe the PTE's and see if there
is enough space and if so just call the multicall 512 times, or perform
a hypercall to setup a super-page. .. But then I wasn't sure how we would
tear down such super-page. |
| Outcomes: Not specified, project outcomes |
|
Time accounting for stolen ticks.
|
Xen Cloud Platform (XCP) and XAPI projects
There are separate wiki pages about XCP and XAPI related projects. Make sure you check these out aswell!
Fuzz testing Xen with Mirage
| Date of insert: 28/11/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Anil Madhavapeddy <anil@recoil.org> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: MirageOS (http://openmirage.org) is a type-safe exokernel written in OCaml which generates highly specialised "appliance" VMs that run directly on Xen without requiring an intervening guest kernel. We would like to use the Mirage/Xen libraries to fuzz test all levels of a typical cloud toolstack. Mirage has low-level bindings for Xen hypercalls, mid-level bindings for domain management, and high-level bindings to XCP for cluster management. This project would build a QuickCheck-style fuzzing mechanism that would perform millions of random operations against a real cluster, and identify bugs with useful backtraces. |
| Outcomes: Not specified, project outcomes |
|
Mirage OS XCP/Xen support
| Date of insert: 28/11/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Anil Madhavapeddy <anil@recoil.org> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: MirageOS (http://openmirage.org) is a type-safe exokernel written in OCaml which generates highly specialised "appliance" VMs that run directly on Xen without requiring an intervening kernel. We would like to explore several strategies for distributed computation using these microkernels, and this project covers writing API bindings for XCP in OCaml. Optionally, we would also like bindings to popular cloud APIs such as Openstack and Amazon EC2.
These API bindings can be used to provide operating-system-level abstractions to the exokernels. For example, hotplugging a vCPU would perform a "VM create" at the XCP level, and make the extra process known to the Mirage runtime so that it can be scheduled for computation. We should be able to spin up 1000s of "CPUs" by using such APIs in a cluster environment. |
| Outcomes: Not specified, project outcomes |
|
From simulation to emulation to production: self-scaling apps
| Date of insert: 28/11/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Anil Madhavapeddy <anil@recoil.org> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: MirageOS (http://openmirage.org) is a type-safe exokernel written in OCaml which generates highly specialised "appliance" VMs that run directly on Xen without requiring an intervening guest kernel. An interesting consequence of programming Mirage applications in a functional language is that the device drivers can be substituted with emulated equivalents. Therefore, it should be possible to test an application under extreme load conditions as a simulation, and then recompile the *same* code into production. The simulation can inject faults and test data structures under distributed conditions, but using a fraction of the resources required for a real deployment.
This project will require a solid grasp of distributed protocols, and functional programming. Okasaki's book will be a useful resource... |
| Outcomes: Not specified, project outcomes |
|
Towards a multi-language unikernel substrate for Xen
| Date of insert: 28/11/2012; Verified: Not updated in 2020; GSoC: Unknown |
| Technical contact: Anil Madhavapeddy <anil@recoil.org> |
| Mailing list/forum for project: xen-devel@ |
| IRC channel for project: #xen-devel |
| Difficulty: Unknown |
| Skills Needed: Unknown |
| Description: There are several languages available that compile directly to Xen microkernels, instead of running under an intervening guest OS. We're dubbing such specialised binaries as "unikernels". Examples include:
Each of these is in a different state of reliability and usability. We would like to survey all of them, build some common representative benchmarks to evaluate them, and build a common toolchain based on XCP that will make it easier to share code across such efforts. This project will require a reasonable grasp of several programming languages and runtimes, and should be an excellent project to learn more about the innards of popular languages. |
| Outcomes: Not specified, project outcomes |
|
Quick links to changelogs of the various Xen related repositories/trees
Please see XenRepositories wiki page!
