Archived/GSoc 2014: Difference between revisions
Lars.kurth (talk | contribs) |
Lars.kurth (talk | contribs) |
||
Line 293: | Line 293: | ||
|Outcomes=Code is submitted to xen-devel@xen.org for inclusion in xen-unstable |
|Outcomes=Code is submitted to xen-devel@xen.org for inclusion in xen-unstable |
||
|GSoC=yes}} |
|GSoC=yes}} |
||
=== Xen Hypervisor Userspace Tools === |
|||
{{project |
|||
|Project=Refactor Linux hotplug scripts |
|||
|Date=15/11/2012 |
|||
|Contact=Roger Pau Monné <[mailto:roger.pau@citrix.com roger.pau@citrix.com]> |
|||
|Desc= |
|||
Current Linux hotplug scripts are all entangled, which makes them really difficult to understand or modify. The reason of hotplug scripts is to give end-users the chance to "easily" support different configuration for Xen devices. |
|||
Linux hotplug scripts should be analized, providing a good description of what each hotplug script is doing. After this, scripts should be cleaned, putting common pieces of code in shared files across all scripts. A Coding style should be applied to all of them when the refactoring is finished. |
|||
[[GSoC_2013#linux-hotplug-scripts]] |
|||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=XL to XCP VM motion |
|||
|Date=15/11/12 |
|||
|Contact=[mailto:ian.campbell@citrix.com Ian Campbell] |
|||
|Desc=Currently [[XL|xl]] (the toolstack supplied alongside Xen) and [[XAPI|xapi]] (the XCP toolstack) have very different concepts about domain configuration, disk image storage etc. In the XCP model domain configuration is persistent and stored in a data base while under xl domain configuration is written in configuration files. Likewise disk images are stored as VDIs in Storage Repositories while under xl disk images are simply files or devices in the dom0 filesystem. For more information on xl see [[XL]]. For more information on XCP see [[XCP Overview]]. |
|||
This project is to produce one or more command-line tools which support migrating VMs between these toolstacks. |
|||
One tool should be provided which takes an xl configuration file and details of an XCP pool. Using the XenAPI XML/RPC interface It should create a VM in the pool with a close approximation of the same configuration and stream the configured disk image into a selected Storage Repository. |
|||
A second tool should be provided which performs the opposite operation, i.e. give a reference to a VM residing in an XCP pool it should produce an XL compatible configuration file and stream the disk image(s) our of Xapi into a suitable format. |
|||
These tools could be reasonably bundled as part of either toolstack and by implication could be written in either C, Ocaml or some other suitable language. |
|||
The tool need not operate on a live VM but that could be considered a stretch goal. |
|||
An acceptable alternative to the proposed implementation would be to implement a tool which converts between a commonly used VM container format which is supported by XCP (perhaps [http://en.wikipedia.org/wiki/Open_Virtualization_Format OVF] or similar) and the xl toolstack configuration file and disk image formats. |
|||
[[GSoC_2013#xl-to-xcp-vm-motion]] |
|||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=VM Snapshots |
|||
|Date=16/01/2013 |
|||
|Contact=<[mailto:stefano.stabellini@eu.citrix.com Stefano Stabellini]> |
|||
|Desc=Although xl is capable of saving and restoring a running VM, it is not currently possible to create a snapshot of the disk together with the rest of the VM. |
|||
QEMU is capable of creating, listing and deleting disk snapshots on QCOW2 and QED files, so even today, issuing the right commands via the QEMU monitor, it is possible to create disk snapshots of a running Xen VM. xl and libxl don't have any knowledge of these snapshots, don't know how to create, list or delete them. |
|||
This project is about implementing disk snapshots support in libxl, using the QMP protocol to issue commands to QEMU. Users should be able to manage the entire life-cycle of their disk snapshots via xl. The candidate should also explore ways to integrate disk snapshots into the regular Xen save/restore mechanisms and provide a solid implementation for xl/libxl. |
|||
[[GSoC_2013#vm-snapshots]] |
|||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=Allowing guests to boot with a passed-through GPU as the primary display |
|||
|Date=01/22/2013 |
|||
|Contact=George Dunlap <george.dunlap@eu.citrix.com> |
|||
|Desc= |
|||
One of the primary drivers of Xen in the "consumer market" of the open-source world is the ability to |
|||
pass through GPUs to guests -- allowing people to run Linux as their main desktop but easily play |
|||
games requiring proprietary operating systems without rebooting. |
|||
GPUs can be easily passed through to guests as secondary displays, but as of yet cannot be passed |
|||
through as primary displays. The main reason is the lack of ability to load the VGA BIOS from the card into the guest. |
|||
The purpose of this project would be to allow HVM guests to load the physical card's VGA bios, so that the guest can |
|||
boot with it as the primary display. |
|||
[[GSoC_2013#gpu-passthrough]] |
|||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=Advanced Scheduling Parameters |
|||
|Date=01/22/2013 |
|||
|Contact=George Dunlap <george.dunlap@eu.citrix.com> |
|||
|Desc= |
|||
The credit scheduler provides a range of "knobs" to control guest behavior, including CPU weight and caps. However, |
|||
a number of users have requested the ability to encode more advanced scheduling logic. For instance, "Let this VM max out for 5 minutes out of any given hour; but after that, impose a cap of 20%, so that even if the system is idle he can't an unlimited amount of CPU power without paying for a higher level of service." |
|||
This is too coarse-grained to do inside the hypervisor; a user-space tool would be sufficient. The goal of this project would |
|||
be to come up with a good way for admins to support these kinds of complex policies in a simple and robust way. |
|||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=Lazy restore using memory paging |
|||
|Date=01/20/2014 |
|||
|Contact=Andres Lagar-Cavilla <andres@lagarcavilla.org> |
|||
|GSoC=Yes |
|||
|Desc=VM Save/restore results in a boatload of IO and non-trivial downtime as the entire memory footprint of a VM is read from IO. |
|||
Xen memory paging support in x86 is now mature enough to allow for lazy restore, whereby the footprint of a VM is backfilled while the VM executes. If the VM hits a page not yet present, it is eagerly paged in. |
|||
There has been some concern recently about the lack of docs and/or mature tools that use xen-paging. This is a good way to address the problem. |
|||
|Skills= A good understanding of save/restore, and virtualized memory management (e.g. EPT, shadow page tables, etc). In principle the entire project can be implemented in user-space C code, but it may be the case that new hypercalls are needed for performance reasons. |
|||
|Outcomes=Expected outcome: |
|||
* Mainline patches for libxc and libxl |
|||
}} |
|||
{{project |
|||
|Project=CPUID Programming for Humans |
|||
|Date=02/04/2014 |
|||
|Contact=Andres Lagar-Cavilla <andres@lagarcavilla.org> |
|||
|GSoC=Yes |
|||
|Desc=When creating a VM, a policy is applied to mask certain CPUID features. Right now it's black magic. |
|||
The KVM stack has done an excellent job of making this human-useable, and understandable. |
|||
For example, in a qemu-kvm command-line you may encounter: |
|||
-cpu SandyBridge,+pdpe1gb,+osxsave,+dca,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme |
|||
And in <qemu>/target-i386.c you find a fairly comprehensive description of x86 processor models, what CPUID features are inherent, and what CPUID feature each of these symbolic flags enables. |
|||
In the Xen world, there is a libxc interface to do the same, although it's all hex and register driven. It's effective, yet horrible to use. |
|||
An ideal outcome would have libxl config files and command line absorb a similarly human-friendly description of the CPUID features a user wishes for the VM, and interface appropriately with libxl. Further, autodetection of best CPUID shuold yield a human-readable output to be able to easily understand what the VM thinks about its processor. |
|||
Finally, interfacing with libvirt should be carefully considered. |
|||
CPUID management is crucial in a heterogeneous cluster where migrations and save restore require careful processor feature selection to avoid blow-ups. |
|||
|Skills= A good understanding of C user-land programming, and the ability to dive into qemu/libvirt (for reference code and integration), as well as libxc and libxl (for implementation). |
|||
|Outcomes=Expected outcome: |
|||
* Mainline patches for libxl |
|||
}} |
|||
=== XAPI === |
=== XAPI === |
Revision as of 01:49, 14 February 2014
The Xen Project is a Linux Foundation collaborative project that develops the
- Xen Hypervisor (for x86 and ARM)
- The XAPI toolstack
- Mirage OS
The project also has excellent relationships with its upstreams (Linux Kernel, the BSDs, QEMU and other projects) and upstreams such as Linux distributions. This is reflected in the project list, which contains many interesting cross-project development projects for students.
GSoC and Xen
This page is used to list project ideas for Google Summer of Code (GSOC) 2014.
Key GSoC resources
Google Summer of Code 2014 is On (see [1]). The Xen Project has applied as a Mentoring Organization. Stay posted.
Finding a project that fits you
This page lists Xen Project development projects for GSoC that can be picked up by anyone! If you're interesting in hacking Xen Project code and want to become a part of our friendly developer community this is the place to start! Ready for the challenge?
To work on a project:
- Find a project that looks interesting (or a bug if you want to start with something simple)
- Send an email to the relevant mailing list (see Developer Mailing Lists) and let us know if you are interested in starting to work or applying on a specific project.
- Post your ideas, questions, RFCs to the relevant mailing list sooner than later so you can get comments and feedback.
You have your own project idea: no problem!
- If you have your own project idea, outline what you are trying to do on the mailing list. If you know the right list, post your project idea on mailing list. Failing that post on xen-devel and we can redirect you to the right list. Make sure you add GSoC 2014 to the subject line.
It is a good idea to ...
The Xen Project has also participated in the Gnome Outreach Program for Women (OPW) in the past. One of the things we learned by participating in OPW is that you will be more successful, happier and get more out of participating in student programs such as GSoC, if you do a bit of prep-work before writing an application. Here is some stuff you can do:
- Contact your mentor early and get to know him or her
- If the Xen Project is accepted into GSoC, start hanging out on our IRC channel. You can use the #xen-opw IRC channel on freenode.net for now (if accepted, we will create a GSoC channel)
- You may want to ask the mentor for a couple of small bitesize work-items (such as reviewing someones patch, a bitesize bug, ...) and start communicating on the relevant mailing list. That helps you become familiar with our development process, the mentor and other community members and will help you chose the right project and help you decide whether the Xen project is for you.
- Note that quite a few Xen maintainers used to be GSoC students once. Feel free to ask community dot manager at xenproject dot org to put you in touch with them if you have questions about their experience.
- Any work you submit before applying for a project should be based on xen-unstable development tree, if the project is Xen Hypervisor and/or tools related. Linux kernel related patches should be based on upstream kernel.org Linux git tree (latest version). XAPI and Mirage OS patches should be based on the right codeline too. Check out the navigation by audience section on the left to find resources.
More resources
Quick links to changelogs of the various Xen related repositories/trees: Please see XenRepositories wiki page!
Before to submit patches, please look at Submitting Xen Patches wiki page and the relevant Xen Project team page. This will contain more information.
If you have new ideas, suggestions or development plans let us know and we'll update this list!
Applying for GSoC
![]() |
Note that we will update this section when more student information on melange is available, to make it easier for you to find information. And of course assuming that the Xen Project will be accepted into GSoC. |
To apply for a project, follow the steps outlined on
- melange
- We do have our own GSoC Student Application Template form
List of peer reviewed Projects
Domain support (PVOPS and Linux)
OVMF Compatibility Support Module support in Xen
|
Parallel xenwatch kthread
|
Utilize Intel QuickPath on network and block path.
|
Enabling the 9P File System transport as a paravirt device
|
Xen Hypervisor Userspace Tools
CPU/RAM/PCI diagram tool
|
KDD (Windows Debugger Stub) enhancements
|
Mirage OS
Create a tiny VM for easy load testing
|
Fuzz testing Xen with Mirage
|
Mirage OS cloud API support
|
List of projects that need more work
Domain support (PVOPS and Linux)
Implement Xen PVSCSI support in xl/libxl toolstack
|
Implement Xen PVUSB support in xl/libxl toolstack
|
Block backend/frontend improvements
|
Xen Hypervisor
Introducing PowerClamp-like driver for Xen
|
Integrating NUMA and Tmem
|
HVM per-event-channel interrupts
|
Xen Hypervisor Userspace Tools
Refactor Linux hotplug scripts
|
XL to XCP VM motion
|
VM Snapshots
|
Allowing guests to boot with a passed-through GPU as the primary display
|
Advanced Scheduling Parameters
|
Lazy restore using memory paging
|
CPUID Programming for Humans
|
XAPI
DRBD Integration
|
Conventions for Projects and Project Mentors
Rules and Advice for Adding Ideas
- Be creative
- Add projects into Project Ideas that Need Review or more work.
- Use the {{GSoC Project}} template to encode ideas on this page. Please read the Template Documentation before you do so.
- Be specific: what do you want to be implemented; if at all possible provide an indication of size and complexity as described above to make it easier for a student to choose ideas
- Check that the project meets the GSoC Program Goals
- If you are willing to mentors those ideas, add your name and email to the idea.
- If you're an interested student, add your name and email next to the idea. It is ok to have several students interested by one idea.
- Aspiring students need to get in touch with the xen.org community manager via community.manager@xen.org to register their interest
Peer Review Goals
We strongly recommend and invite project proposers and project mentors to review each others proposals. When you review, please look out for
- Can a student get going and started with the information in the project description
- Are any unstated assumptions in the proposal, is there undefined terminology, etc. in the proposal
- Can the project completed in 3 months (assume that one month is needed for preparation)
- Does the project meet Google Summer of Code goals, which are
- Create and release open source code for the benefit of all
- Inspire young developers to begin participating in open source development
- Help open source projects identify and bring in new developers and committers
- Provide students the opportunity to do work related to their academic pursuits (think "flip bits, not burgers")
- Give students more exposure to real-world software development scenarios (e.g., distributed development, software licensing questions, mailing-list etiquette)
Peer Review Conventions
The {{GSoC Project}} template used to encode GSoC projects, contains some review functionality. Please read the Template Documentation before you add a template, also please use the conventions below to make comments.
|Review=(delete as addressed) * {{Comment|~~~~:}} Comment 1 * {{Comment|~~~~:}} Comment 2