Archived/GSoc 2014: Difference between revisions
Lars.kurth (talk | contribs) |
Lars.kurth (talk | contribs) No edit summary |
||
Line 116: | Line 116: | ||
}} |
}} |
||
=== Hypervisor === |
=== Xen Hypervisor === |
||
{{InfoLeft|The ones below need work and will be moved into unreviewed, if not improved}} |
|||
{{project |
{{project |
||
Line 149: | Line 150: | ||
|Date=01/30/2013 |
|Date=01/30/2013 |
||
|Contact=Paul Durrant <paul.durrant@citrix.com> |
|Contact=Paul Durrant <paul.durrant@citrix.com> |
||
|Difficulty= |
|||
|Skills=C, some prior knowledge of Xen useful |
|Skills=C, some prior knowledge of Xen useful |
||
|Desc=Windows PV drivers currently have to multiplex all event channel processing onto a single interrupt which is registered with Xen using the HVM_PARAM_CALLBACK_IRQ parameter. This results in a lack of scalability when multiple event channels are heavily used, such as when multiple VIFs in the VM as simultaneously under load. |
|Desc=Windows PV drivers currently have to multiplex all event channel processing onto a single interrupt which is registered with Xen using the HVM_PARAM_CALLBACK_IRQ parameter. This results in a lack of scalability when multiple event channels are heavily used, such as when multiple VIFs in the VM as simultaneously under load. |
||
Line 157: | Line 157: | ||
|GSoC=yes}} |
|GSoC=yes}} |
||
=== Userspace Tools === |
=== Xen Hypervisor Userspace Tools === |
||
{{project |
|||
|Project=CPU/RAM/PCI diagram tool |
|||
|Date=01/30/2013 |
|||
|Contact=Andy Cooper <andrew.cooper3@citrix.com> |
|||
|Difficulty=Low to medium |
|||
|Skills=Linux scripting; basic understanding of PC server hardware |
|||
|Desc=It is often useful in debugging kernel, hypervisor or performance problems to understand the bus topology of a server. This project will create a layout diagram for a server automatically using data from ACPI Tables, SMBios Tables, lspci output etc. This tool would be useful in general Linux environments including Xen and KVM based virtualisation systems. |
|||
There are many avenues for extension such as labelling relevant hardware errata, performing bus throughput calculations etc. |
|||
|Outcomes=A tool is created that can either run on a live Linux system or offline using captured data to produce a graphical representation of the hardware topology of the system including bus topology, hardware device locations, memory bank locations, etc. The tool would be submitted to a suitable open-source project such as the Xen hypervisor project or XCP. |
|||
|GSoC=yes}} |
|||
{{project |
|||
|Project=KDD (Windows Debugger Stub) enhancements |
|||
|Date=01/30/2013 |
|||
|Contact=Paul Durrant <paul.durrant@citrix.com> |
|||
|Difficulty=Medium |
|||
|Skills=C, Kernel Debuggers, Xen, Windows |
|||
|Desc=kdd is a Windows Debugger Stub for Xen hypervisor. It is OSS found under http://xenbits.xen.org/hg/xen-unstable.hg/tools/debugger/kdd |
|||
kdd allows you to debug a running Windows virtual machine on Xen using standard Windows kernel debugging tools like WinDbg. kdd is an external debugger stub for the windows kernel. |
|||
Windows can be debugged without enabling the debugger stub inside windows kernel by using kdd. This is important for debugging hard to reproduce problems on Windows virtual machines that may not have debugging enabled. |
|||
Expected Results: |
|||
# Add support for Windows 8 (x86, x64) to kdd |
|||
# Add support for Windows Server 2012 to kdd |
|||
# Enhance kdd to allow WinDbg to write out usable Windows memory dumps (via .dump debugger extension) for all supported versions |
|||
# Produce a user guide for kdd on Xen wiki page |
|||
Nice to have: Allow kdd to operate on a Windows domain checkpoint file (output of xl save for e.g.) |
|||
|Outcomes=Code is submitted to xen-devel@xen.org for inclusion in the xen-unstable project. |
|||
|GSoC=yes}} |
|||
{{InfoLeft|The ones below need work and will be moved into unreviewed, if not improved}} |
|||
{{project |
{{project |
||
Line 241: | Line 274: | ||
|GSoC=Yes |
|GSoC=Yes |
||
}} |
}} |
||
{{project |
|||
|Project=CPU/RAM/PCI diagram tool |
|||
|Date=01/30/2013 |
|||
|Contact=Andy Cooper <andrew.cooper3@citrix.com> |
|||
|Difficulty=Low to medium |
|||
|Skills=Linux scripting; basic understanding of PC server hardware |
|||
|Desc=It is often useful in debugging kernel, hypervisor or performance problems to understand the bus topology of a server. This project will create a layout diagram for a server automatically using data from ACPI Tables, SMBios Tables, lspci output etc. This tool would be useful in general Linux environments including Xen and KVM based virtualisation systems. |
|||
There are many avenues for extension such as labelling relevant hardware errata, performing bus throughput calculations etc. |
|||
|Outcomes=A tool is created that can either run on a live Linux system or offline using captured data to produce a graphical representation of the hardware topology of the system including bus topology, hardware device locations, memory bank locations, etc. The tool would be submitted to a suitable open-source project such as the Xen hypervisor project or XCP. |
|||
|GSoC=yes}} |
|||
{{project |
|||
|Project=KDD (Windows Debugger Stub) enhancements |
|||
|Date=01/30/2013 |
|||
|Contact=Paul Durrant <paul.durrant@citrix.com> |
|||
|Difficulty=Medium |
|||
|Skills=C, Kernel Debuggers, Xen, Windows |
|||
|Desc=kdd is a Windows Debugger Stub for Xen hypervisor. It is OSS found under http://xenbits.xen.org/hg/xen-unstable.hg/tools/debugger/kdd |
|||
kdd allows you to debug a running Windows virtual machine on Xen using standard Windows kernel debugging tools like WinDbg. kdd is an external debugger stub for the windows kernel. |
|||
Windows can be debugged without enabling the debugger stub inside windows kernel by using kdd. This is important for debugging hard to reproduce problems on Windows virtual machines that may not have debugging enabled. |
|||
Expected Results: |
|||
# Add support for Windows 8 (x86, x64) to kdd |
|||
# Add support for Windows Server 2012 to kdd |
|||
# Enhance kdd to allow WinDbg to write out usable Windows memory dumps (via .dump debugger extension) for all supported versions |
|||
# Produce a user guide for kdd on Xen wiki page |
|||
Nice to have: Allow kdd to operate on a Windows domain checkpoint file (output of xl save for e.g.) |
|||
|Outcomes=Code is submitted to xen-devel@xen.org for inclusion in the xen-unstable project. |
|||
|GSoC=yes}} |
|||
{{project |
{{project |
||
Line 298: | Line 299: | ||
|Desc=When creating a VM, a policy is applied to mask certain CPUID features. Right now it's black magic. |
|Desc=When creating a VM, a policy is applied to mask certain CPUID features. Right now it's black magic. |
||
The KVM stack has done an excellent job of making this human-useable, and understandable. |
|||
For example, in a qemu-kvm command-line you may encounter: |
For example, in a qemu-kvm command-line you may encounter: |
||
Line 320: | Line 321: | ||
}} |
}} |
||
=== Mirage |
=== Mirage OS === |
||
There are separate wiki pages about XCP and XAPI related projects. Make sure you check these out aswell! |
|||
{{project |
{{project |
||
Line 337: | Line 337: | ||
2. at least 2 I/O traces, one for Windows boot and one for Linux boot (any version) |
2. at least 2 I/O traces, one for Windows boot and one for Linux boot (any version) |
||
|GSoC=yes}} |
|GSoC=yes}} |
||
{{project |
{{project |
||
Line 370: | Line 369: | ||
}} |
}} |
||
=== XAPI === |
|||
{{project |
|||
|Project=From simulation to emulation to production: self-scaling apps |
|||
|Date=28/11/2012 |
|||
|Contact=Anil Madhavapeddy <anil@recoil.org> |
|||
|Difficulty=hard |
|||
|Skills=OCaml |
|||
|Desc= |
|||
MirageOS (http://openmirage.org) is a type-safe exokernel written in OCaml which generates highly specialised "appliance" VMs that run directly on Xen without requiring an intervening guest kernel. An interesting consequence of programming Mirage applications in a functional language is that the device drivers can be substituted with emulated equivalents. Therefore, it should be possible to test an application under extreme load conditions as a simulation, and then recompile the *same* code into production. The simulation can inject faults and test data structures under distributed conditions, but using a fraction of the resources required for a real deployment. |
|||
The first task is to familiarise yourself with a typical Mirage application, I suggest a webserver (see https://github.com/mirage/mirage-www). The second task is to replace the ethernet driver with a synthetic equivalent, so we can feed it simulated traffic. Third, we should inject simulated web traffic (recorded from a real session) and attempt to determine how the application response time varies with load (number of connections; incoming packet rate). |
|||
This project will require a solid grasp of distributed protocols, and functional programming. Okasaki's book will be a useful resource... |
|||
|Outcomes=1. a repo/branch with a fake ethernet device and a traffic simulator; 2. an interesting performance graph |
|||
|GSoC=no, too much work |
|||
}} |
|||
{{project |
|||
|Project=Towards a multi-language unikernel substrate for Xen |
|||
|Date=28/11/2012 |
|||
|Contact=Anil Madhavapeddy <anil@recoil.org> |
|||
|Difficulty=hard |
|||
|Skills=OCaml, Haskell, Java |
|||
|Desc= |
|||
There are several languages available that compile directly to Xen microkernels, instead of running under an intervening guest OS. We're dubbing such specialised binaries as "unikernels". Examples include: |
|||
* OCaml: Mirage http://openmirage.org |
|||
* Haskell: HalVM https://github.com/GaloisInc/HaLVM#readme |
|||
* Erlang: ErlangOnXen http://erlangonxen.org |
|||
* Java: GuestVM http://labs.oracle.com/projects/guestvm/, OSv https://github.com/cloudius-systems/osv |
|||
Each of these is in a different state of reliability and usability. We would like to survey all of them, build some common representative benchmarks to evaluate them, and build a common toolchain based on XCP that will make it easier to share code across such efforts. This project will require a reasonable grasp of several programming languages and runtimes, and should be an excellent project to learn more about the innards of popular languages. |
|||
[[GSoC_2013#unikernel-substrate]] |
|||
|Outcomes=1. a repo containing a common library of low-level functions; 2. a proof of concept port of at least 2 systems to this new library |
|||
|GSoC=no, too difficult |
|||
}} |
|||
{{project |
{{project |
||
Line 418: | Line 382: | ||
DRBD should be implemented as a new SR type on top of LVM. The tools for managing DRBD devices need to be built into storage management, along with the logic for switching the active and standby nodes. |
DRBD should be implemented as a new SR type on top of LVM. The tools for managing DRBD devices need to be built into storage management, along with the logic for switching the active and standby nodes. |
||
}} |
}} |
||
* XCP and XAPI development projects: [[XAPI project suggestions]] |
|||
* XCP short-term roadmap: [[XCP short term roadmap]] |
|||
* XCP monthly developer meetings: [[XCP Monthly Meetings]] |
|||
* XAPI developer guide: [[XAPI Developer Guide]] |
|||
Revision as of 01:44, 14 February 2014
The Xen Project is a Linux Foundation collaborative project that develops the
- Xen Hypervisor (for x86 and ARM)
- The XAPI toolstack
- Mirage OS
The project also has excellent relationships with its upstreams (Linux Kernel, the BSDs, QEMU and other projects) and upstreams such as Linux distributions. This is reflected in the project list, which contains many interesting cross-project development projects for students.
GSoC and Xen
This page is used to list project ideas for Google Summer of Code (GSOC) 2014.
Key GSoC resources
Google Summer of Code 2014 is On (see [1]). The Xen Project has applied as a Mentoring Organization. Stay posted.
Finding a project that fits you
This page lists Xen Project development projects for GSoC that can be picked up by anyone! If you're interesting in hacking Xen Project code and want to become a part of our friendly developer community this is the place to start! Ready for the challenge?
To work on a project:
- Find a project that looks interesting (or a bug if you want to start with something simple)
- Send an email to the relevant mailing list (see Developer Mailing Lists) and let us know if you are interested in starting to work or applying on a specific project.
- Post your ideas, questions, RFCs to the relevant mailing list sooner than later so you can get comments and feedback.
You have your own project idea: no problem!
- If you have your own project idea, outline what you are trying to do on the mailing list. If you know the right list, post your project idea on mailing list. Failing that post on xen-devel and we can redirect you to the right list. Make sure you add GSoC 2014 to the subject line.
It is a good idea to ...
The Xen Project has also participated in the Gnome Outreach Program for Women (OPW) in the past. One of the things we learned by participating in OPW is that you will be more successful, happier and get more out of participating in student programs such as GSoC, if you do a bit of prep-work before writing an application. Here is some stuff you can do:
- Contact your mentor early and get to know him or her
- If the Xen Project is accepted into GSoC, start hanging out on our IRC channel. You can use the #xen-opw IRC channel on freenode.net for now (if accepted, we will create a GSoC channel)
- You may want to ask the mentor for a couple of small bitesize work-items (such as reviewing someones patch, a bitesize bug, ...) and start communicating on the relevant mailing list. That helps you become familiar with our development process, the mentor and other community members and will help you chose the right project and help you decide whether the Xen project is for you.
- Note that quite a few Xen maintainers used to be GSoC students once. Feel free to ask community dot manager at xenproject dot org to put you in touch with them if you have questions about their experience.
- Any work you submit before applying for a project should be based on xen-unstable development tree, if the project is Xen Hypervisor and/or tools related. Linux kernel related patches should be based on upstream kernel.org Linux git tree (latest version). XAPI and Mirage OS patches should be based on the right codeline too. Check out the navigation by audience section on the left to find resources.
More resources
Quick links to changelogs of the various Xen related repositories/trees: Please see XenRepositories wiki page!
Before to submit patches, please look at Submitting Xen Patches wiki page and the relevant Xen Project team page. This will contain more information.
If you have new ideas, suggestions or development plans let us know and we'll update this list!
Applying for GSoC
![]() |
Note that we will update this section when more student information on melange is available, to make it easier for you to find information. And of course assuming that the Xen Project will be accepted into GSoC. |
To apply for a project, follow the steps outlined on
- melange
- We do have our own GSoC Student Application Template form
List of peer reviewed Projects
Domain support (PVOPS and Linux)
OVMF Compatibility Support Module support in Xen
|
Parallel xenwatch kthread
|
Utilize Intel QuickPath on network and block path.
|
Enabling the 9P File System transport as a paravirt device
|
Xen Hypervisor
Introducing PowerClamp-like driver for Xen
|
Integrating NUMA and Tmem
|
HVM per-event-channel interrupts
|
Xen Hypervisor Userspace Tools
CPU/RAM/PCI diagram tool
|
KDD (Windows Debugger Stub) enhancements
|
Refactor Linux hotplug scripts
|
XL to XCP VM motion
|
VM Snapshots
|
Allowing guests to boot with a passed-through GPU as the primary display
|
Advanced Scheduling Parameters
|
Lazy restore using memory paging
|
CPUID Programming for Humans
|
Mirage OS
Create a tiny VM for easy load testing
|
Fuzz testing Xen with Mirage
|
Mirage OS cloud API support
|
XAPI
DRBD Integration
|
List of projects that need more work
Domain support (PVOPS and Linux)
Implement Xen PVSCSI support in xl/libxl toolstack
|
Implement Xen PVUSB support in xl/libxl toolstack
|
Block backend/frontend improvements
|
Conventions for Projects and Project Mentors
Rules and Advice for Adding Ideas
- Be creative
- Add projects into Project Ideas that Need Review or more work.
- Use the {{GSoC Project}} template to encode ideas on this page. Please read the Template Documentation before you do so.
- Be specific: what do you want to be implemented; if at all possible provide an indication of size and complexity as described above to make it easier for a student to choose ideas
- Check that the project meets the GSoC Program Goals
- If you are willing to mentors those ideas, add your name and email to the idea.
- If you're an interested student, add your name and email next to the idea. It is ok to have several students interested by one idea.
- Aspiring students need to get in touch with the xen.org community manager via community.manager@xen.org to register their interest
Peer Review Goals
We strongly recommend and invite project proposers and project mentors to review each others proposals. When you review, please look out for
- Can a student get going and started with the information in the project description
- Are any unstated assumptions in the proposal, is there undefined terminology, etc. in the proposal
- Can the project completed in 3 months (assume that one month is needed for preparation)
- Does the project meet Google Summer of Code goals, which are
- Create and release open source code for the benefit of all
- Inspire young developers to begin participating in open source development
- Help open source projects identify and bring in new developers and committers
- Provide students the opportunity to do work related to their academic pursuits (think "flip bits, not burgers")
- Give students more exposure to real-world software development scenarios (e.g., distributed development, software licensing questions, mailing-list etiquette)
Peer Review Conventions
The {{GSoC Project}} template used to encode GSoC projects, contains some review functionality. Please read the Template Documentation before you add a template, also please use the conventions below to make comments.
|Review=(delete as addressed) * {{Comment|~~~~:}} Comment 1 * {{Comment|~~~~:}} Comment 2