Archived/Xen Development Projects: Difference between revisions
m (→Hypervisor) |
|||
Line 204: | Line 204: | ||
and to make it interface as well as possible with the Linux PowerClamp tools, so that the same tools could be used |
and to make it interface as well as possible with the Linux PowerClamp tools, so that the same tools could be used |
||
for both. [[GSoC_2013#powerclamp-for-xen]] |
for both. [[GSoC_2013#powerclamp-for-xen]] |
||
|GSoC=Yes |
|||
}} |
|||
{{project |
|||
|Project=NUMA effects on inter-VM communication and on multi-VM workloads |
|||
|Date=12/12/2012 |
|||
|Contact=Dario Faggioli <[mailto:dario.faggioli@citrix.com dario.faggioli@citrix.com]> |
|||
|Desc=NUMA (Non-Uniform Memory Access) systems are advanced server platforms, comprising multiple ''nodes''. Each node contains processors and memory. An advanced memory controller allows a node to use memory from all other nodes, but that happens, data transfer is slower than accessing local memory. Memory access times are not uniform and depend on the location of the memory and the node from which it is accessed, hence the name. |
|||
If a workload is made up of more than just a VM, running on the same NUMA host, it might be best to have two (or more) VMs share a node, as well as right the opposite, depending on the specific characteristics of he workload itself, and this might be considered during placement, memory migration and perhaps scheduling. |
|||
The idea is that sometimes you have a bunch of VMs that would like to ''cooperate'', and sometimes you have a bunch of VMs that would like to be kept as apart as possible from other VMs (''competitive''). In the ''cooperative'' VMs scenario, one wants to optimize for data flowing across VMs in the same host, e.g., because a lot of data copying is involved (a WebApp and DB VMs working together). This means trying to have VMs sharing data in the same node and, if possible, even in the same PCPU's caches, in order to maximize the memory throughput between the VMs. On the other hand, in the ''competitive'' VMs scenario, one wants to optimize for data flowing between the VMs and outside the host (e.g., when PCI-passthrough is used for NICs). In this case it would be a lot better for these VMs to use memory from different nodes and avoid wasting each other cache lines. |
|||
This project aims at making it possible for the Xen virtualization platform (intended as hypervisor + toolstack) to take advantage of this knowledge about the characteristics of the workload and use it to maximize performances. A first step would be to enhance the automatic NUMA placement algorithm to consider the ''cooperative''-ness and/or the ''competitive''-ness of a VM during placement itself, if provided with such information by the user. A much more complicated development could be to have this relationship between the various running VMs guessed automatically on-line (e.g., by watching the memory mappings and looking for specific patterns), and update the situation accordingly. |
|||
This project fits in the efforts the Xen community is making for improving the performances of Xen on NUMA systems. The full roadmap is available on this Wiki page: [[Xen NUMA Roadmap]] |
|||
|GSoC=Yes |
|GSoC=Yes |
||
}} |
}} |
||
Line 232: | Line 216: | ||
For instance, implementing something like <code>alloc_page_on_any_node_but_the_current_one()</code> (or <code>any_node_except_this_guests_node_set()</code> for multinode guests), and have Xen's Tmem implementation use it (especially in combination with selfballooning), could solve a significant part of the NUMA problem when running Tmem-enabled guests. |
For instance, implementing something like <code>alloc_page_on_any_node_but_the_current_one()</code> (or <code>any_node_except_this_guests_node_set()</code> for multinode guests), and have Xen's Tmem implementation use it (especially in combination with selfballooning), could solve a significant part of the NUMA problem when running Tmem-enabled guests. |
||
}} |
|||
{{project |
|||
|Project=Is Xen ready for the Real-Time/Embedded World? |
|||
|Date=08/08/2012 |
|||
|Contact=Dario Faggioli <[mailto:dario.faggioli@citrix.com dario.faggioli@citrix.com]> |
|||
|Desc= |
|||
No matter if it is to build a [http://gigaom.com/2011/06/25/mobile-virtualization-finds-its-home-in-the-enterprise/ multi-personallity mobile phone], or [http://www.youtube.com/watch?v=j4uMdROzEGI help achieving consolidation in industrial and factory automation], embedded virtualization ([http://en.wikipedia.org/wiki/Embedded_hypervisor [1]], [http://www.ibm.com/developerworks/linux/library/l-embedded-virtualization/index.html [2]], [http://www.wirevolution.com/2012/02/18/mobile-virtualization/ [3]]) is upon us. In fact, quite a number of ''embedded hypervisors'' already exist, e.g.: [http://www.windriver.com/products/hypervisor/ Wind River Hypervisor], [http://dev.b-labs.com/ CodeZero] or [http://www.sysgo.com/products/pikeos-rtos-and-virtualization-concept/ PikeOS]. Xen definitely '''is''' ''small, fast type-1 hypervisor with support for multiple VMs'' [http://en.wikipedia.org/wiki/Embedded_hypervisor [1]], so it could be a good candidate embedded hypervisor. |
|||
In the context of embedded virtualization, providing low latency (when reacting to external events) to the hosted VMs, and a good level of '''temporal isolation''' among the different hosted VMs is something fundamental. This means not only the failures in any VM should not affect any other one, but also that, independently of what a VM is doing, it shouldn't be possible for it to cause any other VM to suffer from higher (and potentially dangerous!) latency and/or impairing the capability of any other VM to respect its own real-time constraints. |
|||
Whether or not Xen is able to provide its (potential) real-time VMs with all this, is something which has not been investigated thoroughly enough yet. This project therefore aims at defining, implementing and executing the proper set of tests, becnhmarks and measuring to understand where Xen stands, identify the sources of inefficiencies and alleviate or remove them. The very first step would be to check how much latency Xen introduces, by running some typical real-time workload within a set of VM, under different host and guest load conditions (e.g., by using [https://rt.wiki.kernel.org/index.php/Cyclictest cyclictest] within the VM themselves). Results can then be compared to what it is achievable with other hypervisors. After this, the code paths contributing most to latency and/or disrupting temporal isolation will need to be identified, and proper solutions to mitigate their effect envisioned and implemented. |
|||
|GSoC=Yes |
|||
}} |
}} |
||
Revision as of 16:03, 10 February 2014
This page lists various Xen related development projects that can be picked up by anyone! If you're interesting in hacking Xen this is the place to start! Ready for the challenge?
To work on a project:
- Find a project that looks interesting (or a bug if you want to start with something simple)
- Send an email to xen-devel mailinglist and let us know you started working on a specific project.
- Post your ideas, questions, RFCs to xen-devel sooner than later so you can get comments and feedback.
- Send patches to xen-devel early for review so you can get feedback and be sure you're going into correct direction.
- Your work should be based on xen-unstable development tree, if it's Xen and/or tools related. After your patch has been merged to xen-unstable it can be backported to stable branches (Xen 4.2, Xen 4.1, etc).
- Your kernel related patches should be based on upstream kernel.org Linux git tree (latest version).
xen-devel mailinglist subscription and archives: http://lists.xensource.com/mailman/listinfo/xen-devel
Before to submit patches, please look at Submitting Xen Patches wiki page.
If you have new ideas, suggestions or development plans let us know and we'll update this list!
List of projects
Domain support
Upstreaming Xen PVSCSI drivers to mainline Linux kernel
|
Upstreaming Xen PVUSB drivers to mainline Linux kernel
|
Implement Xen PVSCSI support in xl/libxl toolstack
|
Implement Xen PVUSB support in xl/libxl toolstack
|
Block backend/frontend improvements
|
Utilize Intel QuickPath on network and block path.
|
perf working with Xen
|
PAT writecombine fixup
|
Parallel xenwatch
|
Enabling the 9P File System transport as a paravirt device
|
OVMF Compatibility Support Module support in Xen
|
Hypervisor
Introducing PowerClamp-like driver for Xen
|
Integrating NUMA and Tmem
|
HVM per-event-channel interrupts
|
Userspace Tools
Refactor Linux hotplug scripts
|
XL to XCP VM motion
|
VM Snapshots
|
Allowing guests to boot with a passed-through GPU as the primary display
|
Advanced Scheduling Parameters
|
CPU/RAM/PCI diagram tool
|
KDD (Windows Debugger Stub) enhancements
|
Lazy restore using memory paging
|
CPUID Programming for Humans
|
Performance
Create a tiny VM for easy load testing
|
Xen Cloud Platform (XCP) and XAPI projects
There are separate wiki pages about XCP and XAPI related projects. Make sure you check these out aswell!
Fuzz testing Xen with Mirage
|
Mirage OS XCP/Xen support
|
From simulation to emulation to production: self-scaling apps
|
Towards a multi-language unikernel substrate for Xen
|
DRBD Integration
|
Expose counters for additional aspects of system performance in XCP
|
Add support for XCP performance counters to be sampled at varying rates
|
XCP backend to Juju/Chef/Puppet/Vagrant
|
RBD (Ceph) client support in XCP
|
Add connection tracking capability to the Linux OVS
|
- XCP and XAPI development projects: XAPI project suggestions
- XCP short-term roadmap: XCP short term roadmap
- XCP monthly developer meetings: XCP Monthly Meetings
- XAPI developer guide: XAPI Developer Guide
Please see XenRepositories wiki page!