Design Sessions 2019

Sessions with published notes

Agreeing priorities for the next year

This is an attempt to agree on the top few (we can decide how many) development and 
community priorities for the next year. We should only include larger feature 
development (that may cover multiple series) with the aim to help code reviewers 
to coordinate review time to get these through the review cycle more quickly.

Attendees are expected to 
a) Propose major developments in the works or pipeline 
b) Vote / provide input on how important these are

For notes, see

• https://cryptpad.fr/pad/#/2/pad/edit/GI1eh1isXzpVQ9lLeHrnIp+n/

osstest before push to nonrewinding branch - aka Branch management

Right now, if a bad commit (that cause osstest test failures) get pushed to 
staging, they get entangled with other work and have to be fixed or reverted. 
Most modern CI systems run tests on proposed branches before those branches are 
pushed to some shared non-rewinding branch.

Can we do the same for Xen and osstest ? How ?

For notes, see

• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00709

Build System gripes

See https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00786

Further defences for speculative sidechannels

The discovery of speculative sidechannels has undermined a lot of the security 
boundaries that software took for granted. Some defences have already been introduced, 
but other areas could do with further hardening. Additionally, we should look for 
ways to reduce the overheads where possible.

Notes:

• see https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00832

Xen Toolstacks

At the moment, we have a binary xl, which can be run; and we have libxl, which links 
against libxc and various other libraries, which must match 100% the hypervisor version. 
We have python and partial golang bindings for some of these libraries, but these may 
break and need recompilation when upgrading to a new version of Xen. This session is 
to discuss what, if anything, to do as a result of this.

A couple of options:

Make a daemon which links against libxl and exposes that functionality in a 
backwards-compatible manner

Make the Xen ABI fully backwards compatible, so that upgrades to Xen will work with 
older libraries

See

• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00789
• https://hackmd.io/0vZaSrKBT2iKWzpVMxDVvQ

Xen Distros

Xen is packaged on several different distributions: CentOS, Debian, Fedora, and 
Arch. This is an opportunity for distro package maintianers (at minimum George 
Dunlap, who maintains the CentOS Xen packages) and distro package users to get 
together and talk about best practices and how things can be improved.

See

• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00791
• https://hackmd.io/vmacVBYbQiORJ9H4_a9Ivw

Live Updating Xen

Live-Updating Xen is replacing the running Xen hypervisor in-place on a system 
without guests noticing.

This feature does not yet exist - it's very early days to get involved and design 
the solution. Following up from the talk on Wednesday, we'll use this slot to talk 
about use-cases, how much and what will be of interest to the community, and 
design discussions on the feature.

For notes, see

• https://cryptpad.fr/pad/#/2/pad/edit/fCwXg1GmSXXG8bc4ridHAsnR/
• https://wiki.xen.org/wiki/Live-Updating_Xen
• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00834

Nested virtualization

Hardware-assisted nested virtualization is becoming more popular and Xen can be 
used in both "L0" and "L1" roles to provide interfaces to open/closed hypervisors 
and guest operating systems. This design session will focus on the long-term interfaces 
necessary to support performant and secure nesting on modern hardware platforms 
and I/O devices.

Related work and Xen Summit talks:

Nested VMX/SVM
PV-Shim
Xen Blanket
uXen (Type-2 Xen)
Xendbg and VMI for nested workloads

For notes, see:

• https://lists.gt.net/xen/devel/560973

Virtio

There is an interest on Arm to support virtio on Xen. This would allow us to 
leverage existing PV protocols (e.g virgil 3d) and offering an easy way for 
users to migrate to Xen.

The topics expected to be discussed during the sessions are:

   - Transport to be used
   - How to prevent backend to access all the guest memory
   - Sketch a plan and potential contributors

For notes, see

• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#01746

Technical debt in the Xen ecosystem (inc libxc/xenstored discussion)

Xen has evolved over time, but there are areas of technical debt which have built 
up and are getting in the way.

For notes, see

• https://hackmd.io/-bCctH_TT4KOy_jF_UIFew

Rust and Xen

Discussing the usage of Rust with Xen. Rust is a safe systems language with a 
focus on zero-cost abstractions. A low level effort is underway to provide 
native bindings to the hypercall ABI to allow native simple recompiling of 
Rust programs as unikernels.

For notes, see

• https://hackmd.io/Bffiwa9BQz2IbIV53h1Q0w

Community Issues / Improvements - Communication, Code of Conduct, etc.

This is a session in which a number of community related issues can should be 
discussed and agreed, for example

- Do we need a Code of Conduct?
- How can we make Xen Project more welcoming for newcomers?
- How do we communicate better and more effectively on the mailing list
- Feedback: we don't set expectations very well (e.g. around cover 
  letters and in other areas)
- We struggle with things such as bikeshedding
- We don't seem to be good at resolving disagreements effectively (even 
  though we have formal mechanisms in place)
- Frequently communication on xen-devel@ comes across as unfriendly: is there a way 
  to do this better? We don't have the same issues on IRC

For notes, see

• Original proposal: https://lists.xenproject.org/archives/html/xen-devel/2019-06/threads.html#01518
• Almost all committers were present
• Agreements
  • Agreed to follow the original proposal which intends to split “unacceptable behaviour” from “aspirational ideal”
  • Agreed to use the LF Events CoC as a baseline - Action on Lars to draft this: see https://docs.google.com/document/d/1rlNWxfcYjkguavNTWoggZhE22n2bxNAuRBHNHE7oTAU/edit?usp=sharing
  • Agreed that this needs to be project wide (at least the CoC)
  • Also agreed to name it CoC (rather than something else as originally proposed by Lars)
    • This requires to change language specific to events
  • An area for discussion which was not quite agreed upon pending an initial proposal was how we would approach the handling of issues
    • A committee
    • Probably 2-3 people of different backgrounds maybe from different subprojects
    • Hidden by an e-mail alias and have to be people which can respond to issues quickly

Documentation improvements

For notes, see https://cryptpad.fr/pad/#/2/pad/view/zZT0PWRUP5cLoRhsHVkZ1NknWjS3gM84Ai6oEYntx58/

LivePatch improvements and features

Development plans for LivePatch on Xen:

Support for module parameters
Additional hooks support
Concept of expectations
inline assembly patching
Replaceable apply/revert actions
Fixes and improvements for stacked modules

For notes, see

• https://lists.xenproject.org/archives/html/xen-devel/2019-07/threads.html#00846

A Journey through Unikraft's Build System

The purpose of this session is to give a tutorial on how to write Unikraft Makefiles and 
Configuration files. This task is needed when developing or porting applications or libraries 
with Unikraft.

See

• File:Unikraft-buildsystem-compressed.pdf

Dom0 Dissagregation with Unikraft

See File:XPDDS19- Unikraft-xensummit-design-session-compressed.pdf

Sessions without published Notes

Exposing hardware-backed CPU timers to limit overhead from Xen's software timers

Problem to Solve
Software-based virtual timers implemented in Xen are a source of overhead and non-determinism 
for virtualized applications. For some industries and use cases, these observables effects 
prevent Xen from being used - performance guarantees and determinism trump almost all other 
matters in some applications.

Ryan Thibodeaux and Christopher Clark seek to host a design session to discuss a proposal 
for exposing hardware-backed CPU timers to guests, with an initial emphasis on Intel CPUs 
and Linux guests. The approach considered would selectively expose the local APIC timers 
in each Intel CPU core, thereby allowing Linux guests to directly utilize high-resolution 
timers in hardware.

The proposed approach would likely entail a new guest configuration option that would control 
access to hardware timers. It is expected that the feature would be available to specific 
configurations where side-effects and guest features are limited, e.g., CPU pinned guests 
using the NULL scheduler and without migration support.

Attendee Contributions
Ryan and Christopher seek feedback and guidance from both the Xen and Linux maintainers. 
Ryan and Christopher will present an initial approach to expose CPU / hardware-backed 
timers (likely including patches for both projects). It is expected that the audience 
will review the design concepts and help to identify risks and limitations of this approach.

Ideally, the design session will conclude with a decision on the feasibility of an 
approach to improve timer performance and identify the configuration options to extend 
or add in support of this approach.

Preparation
Ryan Thibodeaux has already submitted a related patch to the Linux kernel project that 
allows a guest kernel to change (at boot) the minimum timer resolution in the kernel (see https://github.com/torvalds/linux/commit/2ec16bc0fc7ab544f2d405fd4fdd0d717c5ec0c5). 
This mirrors an existing feature in the Xen hypervisor (the "timer_slop" Xen option).

Xen Adventures in Edge Computing

Since Xen's origin in cloud computing, the bare-metal hypervisor has been applied 
to desktops, network middleboxes, vehicles, aerospace, accelerator, graphics and 
other "edge" applications. Vendors have applied Xen in a range of system architectures, 
for performance, security, safety, reliability, power and other axes of optimization.

In long-lived business workflows at the power-constrained edge, domain-specific Xen 
and guest configurations have different priorities than general-purpose Linux 
distributions and cloud platforms. As each vendor's product team earns hard-won 
platform lessons in their domain, how can reusable knowledge be shared with Xen, 
guest and hardware developers in neighboring domains?

Until now, the answer has been fragmentation of the Xen ecosystem, with Xen Summit 
bridging some gaps. Can we borrow from the anti-fragmentation techniques of the 
KVM community, including the rust-vmm "building blocks" approach employed by RedHat, 
AWS, Google and Intel? Can OSS subsets of code, config, policy, build and test 
infrastructures can be shared by multi-domain, Xen-based embedded products?

If you are working with Xen in unusual applications, this session may be of interest.

Run-time control of Speculative mitigation facilities

Instead of existing "spec-ctrl" boot-time cmdline arguments. To be used together 
with Live Microcode update and Live Patching.

A new book on Xen?

The last book about Xen is more than 10 years old. Let's see if there is interest 
for a new book on Xen and if so, what 
sort of content should be expected.

Multi-domain build system

There are multiple build systems proposed to already available that target multi-domain 
builds suitable for use with Xen hypervisor in embedded systems. Still, at least since 
2017, those do not really collaborate and there is no community driven solution exists.

Some time ago we at EPAM systems had a task to create such a tool for Automotive domain, 
that is how our Yocto-based xt-distro appeared.

After using in development environment xt-distro for some time we started facing some 
limitations of our build system, we are thinking about xt-distro v2.0 and would like 
to bring as many interested parties into the design and development as possible, so 
the whole community benefits. This design session will focus on xt-distro, its 
achievements, limitations and ways forward.

Xen hypercall ABI rework

The current hypercall ABI have some issues on Arm that would be warrant for a rework. Some of the issues are:

- Use of guest virtual address is not safe
- Hypercall taking a guest physical frame rather than a full address (problem with mix page granularity)
- A guest can share a page with Xen and another guest (see XSA-295)
During the session, I would expect collect potential other issues and trying to sketch a new ABI.

Xen on RISC-V

Security increasingly depends on hardware, even as we learn the limits of current platforms. 
Open instruction set architectures like RISC-V promise to lower entry costs and accelerate 
hardware innovation, while reducing business overhead. Google's silicon root of trust for 
cloud, Titan is based on RISC-V.

The Linux Foundation CHIPS Alliance supports open-source hardware with high-quality silicon 
IP, open toolchains and well-verified components. The Open Compute Project (OCP) Open 
Domain-Specific Architecture (ODSA) group is defining interfaces to package silicon 
"chiplets" from multiple vendors into domain-specific SoCs.

15 years after inception, the Xen Project stewards a robust, multi-vendor, open-source 
ecosystem for bare-metal virtualization software. Is there room for Xen in the future 
landscape of heterogenous, open-source hardware, including RISC-V platforms?

The RISC-V Hypervisor extension specification is progressing along and hopefully there 
won't be large breaking changes between the current draft version 0.4 and a frozen specification.

Western Digital has been developing a QEMU implementation of the RISC-V Hypervisor 
extension (based on v0.3) and has ported a baremetal Hypervisor called Xvisor. WDC is 
working on a KVM port and has done some work towards a Xen port. WDC and Google are 
both members of https://chipsalliance.org.

Let's discuss how a RISC-V port of Xen can be added to match v0.4 of the evolving 
specification. This will need to include a full port of Xen as well as adding support 
to use the Hypervisor extensions. This must be done with upstreaming in mind, to ensure 
that the RISC-V port will be accepted into mainline Xen, itself a moving target.

(2017) RISC-V Hypervisor extension, https://content.riscv.org/wp-content/uploads/2017/12/Tue0942-riscv-hypervisor-waterman.pdf

(2016) QEMU support for RISC-V, https://www.linux-kvm.org/images/6/6a/02x04B-QEMU-Support_for_the_RISC-V_Instruction_Set_Architecture.pdf