Design Sessions 2019
From Xen
Contents
- 1 Sessions with published notes
- 1.1 Agreeing priorities for the next year
- 1.2 osstest before push to nonrewinding branch - aka Branch management
- 1.3 Build System gripes
- 1.4 Further defences for speculative sidechannels
- 1.5 Xen Toolstacks
- 1.6 Xen Distros
- 1.7 Live Updating Xen
- 1.8 Nested virtualization
- 1.9 Virtio
- 1.10 Technical debt in the Xen ecosystem (inc libxc/xenstored discussion)
- 1.11 Rust and Xen
- 1.12 Community Issues / Improvements - Communication, Code of Conduct, etc.
- 1.13 Documentation improvements
- 1.14 LivePatch improvements and features
- 1.15 A Journey through Unikraft's Build System
- 1.16 Dom0 Dissagregation with Unikraft
- 2 Sessions without published Notes
Sessions with published notes
Agreeing priorities for the next year
This is an attempt to agree on the top few (we can decide how many) development and
community priorities for the next year. We should only include larger feature
development (that may cover multiple series) with the aim to help code reviewers
to coordinate review time to get these through the review cycle more quickly.
Attendees are expected to
a) Propose major developments in the works or pipeline
b) Vote / provide input on how important these are
For notes, see
osstest before push to nonrewinding branch - aka Branch management
Right now, if a bad commit (that cause osstest test failures) get pushed to
staging, they get entangled with other work and have to be fixed or reverted.
Most modern CI systems run tests on proposed branches before those branches are
pushed to some shared non-rewinding branch.
Can we do the same for Xen and osstest ? How ?
For notes, see
Build System gripes
See https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00786
Further defences for speculative sidechannels
The discovery of speculative sidechannels has undermined a lot of the security
boundaries that software took for granted. Some defences have already been introduced,
but other areas could do with further hardening. Additionally, we should look for
ways to reduce the overheads where possible.
Notes:
Xen Toolstacks
At the moment, we have a binary xl, which can be run; and we have libxl, which links
against libxc and various other libraries, which must match 100% the hypervisor version.
We have python and partial golang bindings for some of these libraries, but these may
break and need recompilation when upgrading to a new version of Xen. This session is
to discuss what, if anything, to do as a result of this.
A couple of options:
Make a daemon which links against libxl and exposes that functionality in a
backwards-compatible manner
Make the Xen ABI fully backwards compatible, so that upgrades to Xen will work with
older libraries
See
- https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00789
- https://hackmd.io/0vZaSrKBT2iKWzpVMxDVvQ
Xen Distros
Xen is packaged on several different distributions: CentOS, Debian, Fedora, and
Arch. This is an opportunity for distro package maintianers (at minimum George
Dunlap, who maintains the CentOS Xen packages) and distro package users to get
together and talk about best practices and how things can be improved.
See
- https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00791
- https://hackmd.io/vmacVBYbQiORJ9H4_a9Ivw
Live Updating Xen
Live-Updating Xen is replacing the running Xen hypervisor in-place on a system
without guests noticing.
This feature does not yet exist - it's very early days to get involved and design
the solution. Following up from the talk on Wednesday, we'll use this slot to talk
about use-cases, how much and what will be of interest to the community, and
design discussions on the feature.
For notes, see
- https://cryptpad.fr/pad/#/2/pad/edit/fCwXg1GmSXXG8bc4ridHAsnR/
- https://wiki.xen.org/wiki/Live-Updating_Xen
- https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00834
Nested virtualization
Hardware-assisted nested virtualization is becoming more popular and Xen can be
used in both "L0" and "L1" roles to provide interfaces to open/closed hypervisors
and guest operating systems. This design session will focus on the long-term interfaces
necessary to support performant and secure nesting on modern hardware platforms
and I/O devices.
Related work and Xen Summit talks:
Nested VMX/SVM
PV-Shim
Xen Blanket
uXen (Type-2 Xen)
Xendbg and VMI for nested workloads
For notes, see:
Virtio
There is an interest on Arm to support virtio on Xen. This would allow us to
leverage existing PV protocols (e.g virgil 3d) and offering an easy way for
users to migrate to Xen.
The topics expected to be discussed during the sessions are:
- Transport to be used
- How to prevent backend to access all the guest memory
- Sketch a plan and potential contributors
For notes, see
Technical debt in the Xen ecosystem (inc libxc/xenstored discussion)
Xen has evolved over time, but there are areas of technical debt which have built
up and are getting in the way.
For notes, see
Rust and Xen
Discussing the usage of Rust with Xen. Rust is a safe systems language with a
focus on zero-cost abstractions. A low level effort is underway to provide
native bindings to the hypercall ABI to allow native simple recompiling of
Rust programs as unikernels.
For notes, see
Community Issues / Improvements - Communication, Code of Conduct, etc.
This is a session in which a number of community related issues can should be
discussed and agreed, for example
- Do we need a Code of Conduct?
- How can we make Xen Project more welcoming for newcomers?
- How do we communicate better and more effectively on the mailing list
- Feedback: we don't set expectations very well (e.g. around cover
letters and in other areas)
- We struggle with things such as bikeshedding
- We don't seem to be good at resolving disagreements effectively (even
though we have formal mechanisms in place)
- Frequently communication on xen-devel@ comes across as unfriendly: is there a way
to do this better? We don't have the same issues on IRC
For notes, see
- Original proposal: https://lists.xenproject.org/archives/html/xen-devel/2019-06/threads.html#01518
- Almost all committers were present
- Agreements
- Agreed to follow the original proposal which intends to split “unacceptable behaviour” from “aspirational ideal”
- Agreed to use the LF Events CoC as a baseline - Action on Lars to draft this: see https://docs.google.com/document/d/1rlNWxfcYjkguavNTWoggZhE22n2bxNAuRBHNHE7oTAU/edit?usp=sharing
- Agreed that this needs to be project wide (at least the CoC)
- Also agreed to name it CoC (rather than something else as originally proposed by Lars)
- This requires to change language specific to events
- An area for discussion which was not quite agreed upon pending an initial proposal was how we would approach the handling of issues
- A committee
- Probably 2-3 people of different backgrounds maybe from different subprojects
- Hidden by an e-mail alias and have to be people which can respond to issues quickly
Documentation improvements
For notes, see https://cryptpad.fr/pad/#/2/pad/view/zZT0PWRUP5cLoRhsHVkZ1NknWjS3gM84Ai6oEYntx58/
LivePatch improvements and features
Development plans for LivePatch on Xen:
Support for module parameters
Additional hooks support
Concept of expectations
inline assembly patching
Replaceable apply/revert actions
Fixes and improvements for stacked modules
For notes, see
A Journey through Unikraft's Build System
The purpose of this session is to give a tutorial on how to write Unikraft Makefiles and
Configuration files. This task is needed when developing or porting applications or libraries
with Unikraft.
See
Dom0 Dissagregation with Unikraft
See File:XPDDS19- Unikraft-xensummit-design-session-compressed.pdf
Sessions without published Notes
Exposing hardware-backed CPU timers to limit overhead from Xen's software timers
Problem to Solve
Software-based virtual timers implemented in Xen are a source of overhead and non-determinism
for virtualized applications. For some industries and use cases, these observables effects
prevent Xen from being used - performance guarantees and determinism trump almost all other
matters in some applications.
Ryan Thibodeaux and Christopher Clark seek to host a design session to discuss a proposal
for exposing hardware-backed CPU timers to guests, with an initial emphasis on Intel CPUs
and Linux guests. The approach considered would selectively expose the local APIC timers
in each Intel CPU core, thereby allowing Linux guests to directly utilize high-resolution
timers in hardware.
The proposed approach would likely entail a new guest configuration option that would control
access to hardware timers. It is expected that the feature would be available to specific
configurations where side-effects and guest features are limited, e.g., CPU pinned guests
using the NULL scheduler and without migration support.
Attendee Contributions
Ryan and Christopher seek feedback and guidance from both the Xen and Linux maintainers.
Ryan and Christopher will present an initial approach to expose CPU / hardware-backed
timers (likely including patches for both projects). It is expected that the audience
will review the design concepts and help to identify risks and limitations of this approach.
Ideally, the design session will conclude with a decision on the feasibility of an
approach to improve timer performance and identify the configuration options to extend
or add in support of this approach.
Preparation
Ryan Thibodeaux has already submitted a related patch to the Linux kernel project that
allows a guest kernel to change (at boot) the minimum timer resolution in the kernel (see https://github.com/torvalds/linux/commit/2ec16bc0fc7ab544f2d405fd4fdd0d717c5ec0c5).
This mirrors an existing feature in the Xen hypervisor (the "timer_slop" Xen option).
Xen Adventures in Edge Computing
Since Xen's origin in cloud computing, the bare-metal hypervisor has been applied
to desktops, network middleboxes, vehicles, aerospace, accelerator, graphics and
other "edge" applications. Vendors have applied Xen in a range of system architectures,
for performance, security, safety, reliability, power and other axes of optimization.
In long-lived business workflows at the power-constrained edge, domain-specific Xen
and guest configurations have different priorities than general-purpose Linux
distributions and cloud platforms. As each vendor's product team earns hard-won
platform lessons in their domain, how can reusable knowledge be shared with Xen,
guest and hardware developers in neighboring domains?
Until now, the answer has been fragmentation of the Xen ecosystem, with Xen Summit
bridging some gaps. Can we borrow from the anti-fragmentation techniques of the
KVM community, including the rust-vmm "building blocks" approach employed by RedHat,
AWS, Google and Intel? Can OSS subsets of code, config, policy, build and test
infrastructures can be shared by multi-domain, Xen-based embedded products?
If you are working with Xen in unusual applications, this session may be of interest.
Run-time control of Speculative mitigation facilities
Instead of existing "spec-ctrl" boot-time cmdline arguments. To be used together
with Live Microcode update and Live Patching.
A new book on Xen?
The last book about Xen is more than 10 years old. Let's see if there is interest
for a new book on Xen and if so, what
sort of content should be expected.
Multi-domain build system
There are multiple build systems proposed to already available that target multi-domain
builds suitable for use with Xen hypervisor in embedded systems. Still, at least since
2017, those do not really collaborate and there is no community driven solution exists.
Some time ago we at EPAM systems had a task to create such a tool for Automotive domain,
that is how our Yocto-based xt-distro appeared.
After using in development environment xt-distro for some time we started facing some
limitations of our build system, we are thinking about xt-distro v2.0 and would like
to bring as many interested parties into the design and development as possible, so
the whole community benefits. This design session will focus on xt-distro, its
achievements, limitations and ways forward.
Xen hypercall ABI rework
The current hypercall ABI have some issues on Arm that would be warrant for a rework. Some of the issues are:
- Use of guest virtual address is not safe
- Hypercall taking a guest physical frame rather than a full address (problem with mix page granularity)
- A guest can share a page with Xen and another guest (see XSA-295)
During the session, I would expect collect potential other issues and trying to sketch a new ABI.
Xen on RISC-V
Security increasingly depends on hardware, even as we learn the limits of current platforms.
Open instruction set architectures like RISC-V promise to lower entry costs and accelerate
hardware innovation, while reducing business overhead. Google's silicon root of trust for
cloud, Titan is based on RISC-V.
The Linux Foundation CHIPS Alliance supports open-source hardware with high-quality silicon
IP, open toolchains and well-verified components. The Open Compute Project (OCP) Open
Domain-Specific Architecture (ODSA) group is defining interfaces to package silicon
"chiplets" from multiple vendors into domain-specific SoCs.
15 years after inception, the Xen Project stewards a robust, multi-vendor, open-source
ecosystem for bare-metal virtualization software. Is there room for Xen in the future
landscape of heterogenous, open-source hardware, including RISC-V platforms?
The RISC-V Hypervisor extension specification is progressing along and hopefully there
won't be large breaking changes between the current draft version 0.4 and a frozen specification.
Western Digital has been developing a QEMU implementation of the RISC-V Hypervisor
extension (based on v0.3) and has ported a baremetal Hypervisor called Xvisor. WDC is
working on a KVM port and has done some work towards a Xen port. WDC and Google are
both members of https://chipsalliance.org.
Let's discuss how a RISC-V port of Xen can be added to match v0.4 of the evolving
specification. This will need to include a full port of Xen as well as adding support
to use the Hypervisor extensions. This must be done with upstreaming in mind, to ensure
that the RISC-V port will be accepted into mainline Xen, itself a moving target.
(2017) RISC-V Hypervisor extension, https://content.riscv.org/wp-content/uploads/2017/12/Tue0942-riscv-hypervisor-waterman.pdf
(2016) QEMU support for RISC-V, https://www.linux-kvm.org/images/6/6a/02x04B-QEMU-Support_for_the_RISC-V_Instruction_Set_Architecture.pdf