Difference between revisions of "Archived/Xen Roadmap/4.3"

From Xen
(Proposed timeline)
 
(9 intermediate revisions by one other user not shown)
Line 5: Line 5:
 
* '''Code Freezing point''': 15 April, 2013
 
* '''Code Freezing point''': 15 April, 2013
 
* '''First RC''': 6 May 2013
 
* '''First RC''': 6 May 2013
* '''Release''': 17 June 2013
+
* '''Release''': 2 July 2013
   
 
The RCs and release will of course depend on stability and bugs, and
 
The RCs and release will of course depend on stability and bugs, and
Line 12: Line 12:
 
follows:
 
follows:
   
Last updated 10 May 2013
+
Last updated 27 June 2013
   
 
== Feature list ==
 
== Feature list ==
Line 30: Line 30:
 
=== Completed ===
 
=== Completed ===
 
<pre>
 
<pre>
* Serial console improvements
 
-EHCI debug port
 
 
 
* Default to QEMU upstream (partial)
 
* Default to QEMU upstream (partial)
 
- pci pass-thru (external)
 
- pci pass-thru (external)
Line 38: Line 35:
 
- xl cd-{insert,eject} (external)
 
- xl cd-{insert,eject} (external)
   
  +
* openvswitch toostack integration
* CPUID-based idle (don't rely on ACPI info f/ dom0)
 
  +
To label "tech-preview" unless we get good testing (>10 individuals)
  +
  +
* NUMA scheduler affinity
  +
  +
* Install into /usr/local by default
  +
owner: Ian Campbell
   
 
* Persistent grants for blk (external)
 
* Persistent grants for blk (external)
Line 45: Line 48:
   
 
* Allow XSM to override IS_PRIV checks in the hypervisor
 
* Allow XSM to override IS_PRIV checks in the hypervisor
  +
  +
* vTPM updates
   
 
* Scalability: 16TiB of RAM
 
* Scalability: 16TiB of RAM
   
  +
* CPUID-based idle (don't rely on ACPI info f/ dom0)
* xl QXL Spice support
 
   
  +
* Serial console improvements
* Install into /usr/local by default
 
  +
-EHCI debug port
owner: Ian Campbell
 
 
* openvswitch toostack integration
 
To label "tech-preview" unless we get good testing (>10 individuals)
 
 
* NUMA scheduler affinity
 
 
* ARM v7 server port (basic)
 
* ARM v8 server port (basic)
 
 
* vTPM updates
 
 
</pre>
 
</pre>
   
 
=== Bugs ===
 
=== Bugs ===
 
<pre>
 
<pre>
  +
* qemu-upstream not freeing pirq
* xl, compat mode, and older kernels
 
  +
> http://www.gossamer-threads.com/lists/xen/devel/281498
owner: Wei Liu
 
  +
priority: high, probably not a blocker
Many older 32-bit PV kernels that can run on a 64-bit hypervisor with
 
  +
status: patches posted
xend do not work when started with xl. The following work-around seems to
 
work:
 
xl create -p lightning.cfg
 
xenstore-write /local/domain/$(xl domid lightning)/device/vbd/51713/protocol x86_32-abi
 
xl unpause lightning
 
This node is normally written by the guest kernel, but for older kernels
 
seems not to be. xend must have a work-around; port this work-around to xl.
 
   
  +
* AMD: IOMMU accidental clearing
* AMD NPT performance regression after c/s 24770:7f79475d3de7
 
owner: ?
+
owner: Jan Beulich, Suravee Suthikulpanit
  +
status:
Reference: http://marc.info/?l=xen-devel&m=135075376805215
 
   
  +
* __update_vcpu_system_time if system_time comes out negative
* qemu-upstream: cd-insert and cd-eject not working
 
  +
status: Not for 4.3
http://marc.info/?l=xen-devel&m=135850249808960
 
 
* Revert Jan's debugging patch (commit bd9be94)
 
owner: Jan Beulich
 
 
* Race condition in claim hypercall
 
owner: Ian Jackson, Konrad Wilk
 
 
* Remove hardcoded mobprobe's in xencommons
 
owner: Wei Liu
 
status: ?
 
 
* Windows 2003 fails to install in Xen-unstable tip
 
> Narrowed down to c/s 2fe82ac11fd078485388fe7c5e8bc3b6ac9185b0
 
owner: Jan Beulich
 
 
</pre>
 
</pre>
 
   
 
=== Not yet completed ===
 
=== Not yet completed ===
 
<pre>
 
<pre>
  +
* ARM v7 server port (basic)
* Multi-vector PCI MSI (support at least for Dom0)
 
  +
* ARM v8 server port (basic)
owner: jan@suse
 
  +
- Goal: Good Arndale support for 4.3
status: Patches posted for Intel; AMD not yet done
 
  +
- Regressions here will not slip the release, so development is ongoing
prognosis: Fair
 
 
* xl USB pass-through for HVM guests using Qemu USB emulation
 
owner: George
 
status: v6 patch series posted
 
prognosis: Fair
 
 
* Rationalized backend scripts
 
owner: roger@citrix
 
status: patches posted
 
prognosis: Good
 
 
* Scripts for driver domains (depends on backend scripts)
 
owner: roger@citrix
 
status:
 
prognosis: Fair
 
 
 
</pre>
 
</pre>
 
=== External projects (Not affected by feature freeze) ===
 
=== External projects (Not affected by feature freeze) ===
 
<pre>
 
<pre>
  +
* xl pci-detach broken for pv guests?
  +
> http://bugs.xenproject.org/xen/bug/12
  +
> kernel doesn't know about moving into states 7 and 8
  +
status: External
  +
  +
* Race in PV shutdown between tool detection and shutdown watch
  +
> http://www.gossamer-threads.com/lists/xen/devel/282467
  +
> Nothing to do with ACPI
  +
status: Probably a bug in Linux xen drivers
  +
 
* Multi-page blk rings (external)
 
* Multi-page blk rings (external)
 
- blkback in kernel roger@citrix
 
- blkback in kernel roger@citrix
Line 152: Line 120:
 
=== For future releases ===
 
=== For future releases ===
 
<pre>
 
<pre>
  +
* xl does not handle migrate interruption gracefully
  +
> If you start a localhost migrate, and press "Ctrl-C" in the middle,
  +
> you get two hung domains
  +
status: Probably not for 4.3
  +
  +
* libxl / xl does not handle failure of remote qemu gracefully
  +
> Easiest way to reproduce:
  +
> - set "vncunused=0" and do a local migrate
  +
> - The "remote" qemu will fail because the vnc port is in use
  +
> The failure isn't the problem, but everything being stuck afterwards is
  +
status: Probably not for 4.3
  +
  +
* qxl
  +
> http://bugs.xenproject.org/xen/bug/11http://bugs.xenproject.org/xen/bug/11
  +
- Uninitialized struct element in qemu
  +
- Revert 5479961 to re-enable qxl in xl,libxl
  +
- Option in Xen top-level to enable qxl support in qemu tree
  +
- Fix sse2 MMIO issue
  +
  +
* Polish up xenbugtool
  +
  +
* acpi-related xenstore entries not propagated on migrate
  +
> http://www.gossamer-threads.com/lists/xen/devel/282466
  +
> Only used by hvmloader; only a clean-up, not a bug.
  +
status: Not for 4.3
  +
  +
* mac address changes on reboot if not specified in config file
  +
> Needs a robust way to "add" to the config
  +
status: Too much for 4.3
  +
  +
* Default to credit2
  +
  +
* PVIO NUMA improvements
  +
- NUMA affinity for vcpus
  +
- PV guest NUMA interface
  +
- Sensible dom0 NUMA layout
  +
- Toolstack pinning backend thread / virq to appropraite d0 vcpu
  +
  +
* xenperf
  +
Owner: Boris Otovsky
  +
  +
* libxl config file
  +
  +
* libxl: Don't use RAW format for "URL"-based qdisks (e.g., rbd:rbd/foo.img)
  +
- Figure out whether to use a generic URL or have a specific type for each one
  +
- Check existence of disk file for all RAW
  +
  +
* xl QXL Spice support
  +
  +
* Remove hardcoded mobprobe's in xencommons
  +
owner: Wei Liu
  +
status: still in discussion
  +
  +
* xl USB pass-through for HVM guests using Qemu USB emulation
  +
owner: George
  +
status: v6 patch series posted
  +
prognosis: Fair
  +
  +
* Rationalized backend scripts
  +
owner: roger@citrix
  +
status: patches posted
  +
prognosis: Good
  +
  +
* Scripts for driver domains (depends on backend scripts)
  +
owner: roger@citrix
  +
status:
  +
prognosis: Fair
  +
  +
* Multi-vector PCI MSI (support at least for Dom0)
  +
owner: jan@suse
  +
status: Patches posted for Intel; AMD not yet done
  +
prognosis: Fair
  +
 
* xl: passing more defaults in configuration in xl.conf
 
* xl: passing more defaults in configuration in xl.conf
 
owner: ?
 
owner: ?
Line 283: Line 324:
 
prognosis: ?
 
prognosis: ?
   
  +
* Testing: Network driver domains
* IllumOS (OpenSolaris fork) support
 
  +
owner: Igor Kozhukov
 
  +
* Testing: new libxl w/ previous versions of xl
status: Stopped
 
  +
  +
* Testing: S3
 
</pre>
 
</pre>
   

Latest revision as of 12:49, 12 July 2013

Proposed timeline

We are planning on a 9-month release cycle. Xen 4.2 was released on 17 September 2012. Based on that, below are our estimated dates:

  • Feature Freeze: 25 March, 2013
  • Code Freezing point: 15 April, 2013
  • First RC: 6 May 2013
  • Release: 2 July 2013

The RCs and release will of course depend on stability and bugs, and will therefore be fairly unpredictable. Each new feature will be considered on a case-by-case basis; but the general rule will be as follows:

Last updated 27 June 2013

Feature list

Below is a list of features we're tracking for this release. If you have updates to a tracked feature you are working on, or are working on a feature and wish it to be tracked, please let me know.

NB: Several of the items on this list marked (external). These are not part of the Xen tree, but are directly related to our users' experience (e.g., work in Linux or qemu) or to integration with other important projects (e.g., libvirt indings). Since all of these are part of the Xen community work, and comes from the same pool of labor, it makes sense to track the progress here, even though they won't explicitly be released as part of 4.3.

Meanings of prognoses:

  • Excellent: It would be very unlikely for this not to be finished in time.
  • Good: Everything is on track, and is likely to make it.
  • Fair: A pretty good chance of making it, but not as certain
  • Poor: Likely not to make it unless intervention is made
  • Not for 4.3: Self-explanatory


Completed

* Default to QEMU upstream (partial)
 - pci pass-thru (external)
 - enable dirtybit tracking during migration (external)
 - xl cd-{insert,eject} (external)

* openvswitch toostack integration
  To label "tech-preview" unless we get good testing (>10 individuals)

* NUMA scheduler affinity

* Install into /usr/local by default
  owner: Ian Campbell

* Persistent grants for blk (external)
 - Linux
 - qemu

* Allow XSM to override IS_PRIV checks in the hypervisor

* vTPM updates

* Scalability: 16TiB of RAM

* CPUID-based idle (don't rely on ACPI info f/ dom0)

* Serial console improvements
  -EHCI debug port

Bugs

* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 priority: high, probably not a blocker
 status: patches posted

* AMD: IOMMU accidental clearing
  owner: Jan Beulich, Suravee Suthikulpanit
  status: 

* __update_vcpu_system_time if system_time comes out negative
  status: Not for 4.3

Not yet completed

* ARM v7 server port (basic)
* ARM v8 server port (basic)
 - Goal: Good Arndale support for 4.3
 - Regressions here will not slip the release, so development is ongoing

External projects (Not affected by feature freeze)

* xl pci-detach broken for pv guests?
  > http://bugs.xenproject.org/xen/bug/12
  > kernel doesn't know about moving into states 7 and 8
  status: External

* Race in PV shutdown between tool detection and shutdown watch
 > http://www.gossamer-threads.com/lists/xen/devel/282467
 > Nothing to do with ACPI
 status: Probably a bug in Linux xen drivers

* Multi-page blk rings (external)
 - blkback in kernel roger@citrix
 - qemu blkback
  status: Overall blk architecture being discussed
  prognosis: Fair

* Xen EFI feature: Xen can boot from grub.efi
 owner: Daniel Kiper
 status: Just begun
 prognosis: Fair

* libvirt/libxl integration (external)
 - Update libvirt to 4.2
   status: Patch accepted
 - Migration
   owner: cyliu@suse (?)
   status: first draft implemented, not yet submitted
   prognosis: ?
 - Itemize other things that need work
   To begin with, we need someone to go and make some lists:
   - Features available in libvirt/KVM not available in libvirt/libxl
     See http://libvirt.org/hvsupport.html
   - Features available in xl/Xen but not available in libvirt/Xen

For future releases

* xl does not handle migrate interruption gracefully
  > If you start a localhost migrate, and press "Ctrl-C" in the middle,
  > you get two hung domains
  status: Probably not for 4.3

* libxl / xl does not handle failure of remote qemu gracefully
  > Easiest way to reproduce: 
  >  - set "vncunused=0" and do a local migrate
  >  - The "remote" qemu will fail because the vnc port is in use
  > The failure isn't the problem, but everything being stuck afterwards is
  status: Probably not for 4.3

* qxl
  > http://bugs.xenproject.org/xen/bug/11http://bugs.xenproject.org/xen/bug/11
  - Uninitialized struct element in qemu
  - Revert 5479961 to re-enable qxl in xl,libxl
  - Option in Xen top-level to enable qxl support in qemu tree
  - Fix sse2 MMIO issue

* Polish up xenbugtool

* acpi-related xenstore entries not propagated on migrate
 > http://www.gossamer-threads.com/lists/xen/devel/282466
 > Only used by hvmloader; only a clean-up, not a bug.
 status: Not for 4.3

* mac address changes on reboot if not specified in config file
  > Needs a robust way to "add" to the config
  status: Too much for 4.3

* Default to credit2

* PVIO NUMA improvements
 - NUMA affinity for vcpus
 - PV guest NUMA interface
 - Sensible dom0 NUMA layout
 - Toolstack pinning backend thread / virq to appropraite d0 vcpu

* xenperf
  Owner: Boris Otovsky

* libxl config file

* libxl: Don't use RAW format for "URL"-based qdisks (e.g., rbd:rbd/foo.img)
  - Figure out whether to use a generic URL or have a specific type for each one
  - Check existence of disk file for all RAW

* xl QXL Spice support

* Remove hardcoded mobprobe's in xencommons
  owner: Wei Liu
  status: still in discussion

* xl USB pass-through for HVM guests using Qemu USB emulation
  owner: George
  status: v6 patch series posted
  prognosis: Fair

* Rationalized backend scripts
  owner: roger@citrix
  status: patches posted
  prognosis: Good

* Scripts for driver domains (depends on backend scripts)
  owner: roger@citrix
  status: 
  prognosis: Fair

* Multi-vector PCI MSI (support at least for Dom0)
  owner: jan@suse
  status: Patches posted for Intel; AMD not yet done
  prognosis: Fair

* xl: passing more defaults in configuration in xl.conf
  owner: ?
  There are a number of options for which it might be useful to pass a
  default in xl.conf.  For example, if we could have a default
  "backend" parameter for vifs, then it would be easy to switch back
  and forth between a backend in a driver domain and a backend in dom0.

* xl PVUSB pass-through for PV guests
* xl PVUSB pass-through for HVM guests
  owner: George
  status: ?
  prognosis: Poor
  xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both PV and HVM guests).
  - port the xm/xend functionality to xl.
  - this PVUSB feature does not require support or emulation from Qemu.
  - upstream the Linux frontend/backend drivers. Current work-in-progress versions are in Konrad's git tree.
  - James Harper's GPLPV drivers for Windows include PVUSB frontend drivers.

* qemu-based stubdom (Linux or BSD libc)
   owner: anthony@citrix
   status: in progress
   prognosis: ?
   qemu-upstream needs a more fully-featured libc than exists in
   mini-os.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

* V4V: Inter-domain communication
  owner (Xen): dominic.curran@citrix.com
  status (Xen): patches submitted
  prognosis: Fair
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

* NUMA Memory migration 
  owner: dario@citrix
  status: in progress
  prognosis: Fair

* Event channel scalability
  owner: wei@citrix or david@citrix
  status: RFC v5 submitted
  prognosis: Deciding whether to shoot for 3-level (4.3) or FIFO (4.4) 
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* PVH mode (w/ Linux)
  owner: mukesh@oracle
  status (Linux): 3rd draft patches posted.  
  status (Xen): RFC submitted
  prognosis: Tech preview only

* blktap3
  owner: thanos@citrix
  status: RFCs posted
  prognosis: Not for 4.3

* Guest EFI booting
 - status: tianocore in-tree, some build problems.
   prognosis: Poor.
   Needs new owner.

* Multi-page net protocol (external)
  owner: ijc@citrix or annie.li@oracle
  status: Initial patches posted (by Wei Liu)
  prognosis: Poor
  expand the network ring protocol to allow multiple pages for
  increased throughput

* Wait queues for mm
  owner: ?
  status: Draft posted Feb 2012; more work to do.
  prognosis: Poor

* Xen EFI feature: pvops dom0 able to make use of EFI run-time services (external)
 owner: Daniel Kiper
 status: Just begun
 prognosis: Probably not for 4.3 (?)

* Persistent grants for net
  owner: annie.li@citrix
  status: Initial implementation not getting expected gains; more investigation requried
  prognosis: Probably not for 4.3

* Serial console improvements
  owner: ?
  status: Stalled (see below)
  prognosis: Probably not for 4.3.
  -xHCI debug port (Needs hardware)
  -Firewire (needs hardware)

* Make storage migration possible
  owner: ?
  status: none
  prognosis: Probably delay until 4.4
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: none
  prognosis: Probably delay until 4.4
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: none
  prognosis: Probably need 4.4
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* xl vm-{export,import}
  owner: ?
  status: none
  prognosis: Prob put off until 4.4 (or GSoC project)
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.
  
* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: none
  prognosis: Prob put off until 4.4

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?
  prognosis: ?

* Testing: Network driver domains

* Testing: new libxl w/ previous versions of xl

* Testing: S3