Difference between revisions of "QEMU Upstream"

From Xen
(SPICE / QXL)
m (Fix typo)
 
(26 intermediate revisions by 7 users not shown)
Line 1: Line 1:
  +
= History =
= How to build QEMU with Xen =
 
   
  +
Historically Xen contained a fork of qemu with Xen support added, known as 'qemu-xen-traditional' in the xl toolstack. However since Qemu 1.0 support for Xen has been part of the mainline Qemu and can be used with Xen from 4.2 onwards. The xl toolstack describes this version as 'qemu-xen', and this became the default from Xen 4.3 onward.
Nowadays, xen-unstable will clone and build a qemu git tree from xenbits. (It is qemu-stable-1.0 today). But you can follow this guide to build the upstream unstable qemu git tree.
 
   
  +
The 'qemu-xen-traditional' fork remains available to support guest OSs that were installed using it. After a Dom0 upgrade to a newer Xen version the new 'qemu-xen' device model is recognized as a significant hardware change. Some guest operating systems, especially non-free systems that rely on licensing bound to a specific hardware, do not behave nicely when hardware is changed under their feet.
You can add this to your "xen-unstable.git/.config" file, so xen-unstable will clone and build the upstream tree.
 
  +
  +
= Xen Qemu Trees =
  +
  +
The '''qemu-xen''' code is maintained upstream in the [http://wiki.qemu.org/Main_Page qemu.org] git trees. In addition the Xen project also maintains its own stable branch of qemu, based on the upstream stable branches with a small number of additional fixes for Xen.
  +
These can be found on [http://xenbits.xen.org xenbits] in [https://xenbits.xen.org/gitweb/?p=qemu-xen.git;a=summary qemu-xen.git]
  +
with a branch for each release of Xen named 'stable-VERSION',
  +
e.g. [https://xenbits.xen.org/gitweb/?p=qemu-xen.git;a=shortlog;h=refs/heads/stable-4.10 stable-4.10].
  +
  +
The '''qemu-xen-traditional''' fork is maintained on [http://xenbits.xen.org xenbits]
  +
in [https://xenbits.xen.org/gitweb/?p=qemu-xen-traditional.git;a=summary qemu-xen-traditional.git].
  +
There are branches for each release of Xen named 'stable-VERSION' e.g. [https://xenbits.xen.org/gitweb/?p=qemu-xen-traditional.git;a=shortlog;h=refs/heads/stable-4.10 stable-4.10].
  +
  +
By default the Xen build system will clone and build both versions of QEMU from the branches on [http://xenbits.xen.org xenbits].
  +
  +
=== Obsolete Xen Qemu Trees ===
  +
  +
In the past, we maintained the following trees:
  +
* '''qemu-xen''': The 'qemu-xen' code is maintained upstream in the [http://wiki.qemu.org/Main_Page qemu.org] git trees. In addition the Xen project also maintains its own stable branch of qemu, based on the upstream stable branches with a small number of additional fixes for Xen. These can be found on [http://xenbits.xen.org xenbits] in '''qemu-upstream-VERSION.git''' e.g. [http://xenbits.xen.org/gitweb/?p=qemu-upstream-unstable.git;a=summary qemu-upstream-unstable.git], [http://xenbits.xen.org/gitweb/?p=qemu-upstream-4.3-testing.git;a=summary qemu-upstream-4.3-testing.git].
  +
* '''qemu-xen-traditional''': [http://xenbits.xen.org xenbits] in '''qemu-xen-VERSION.git''' e.g. [http://xenbits.xen.org/gitweb/?p=qemu-xen-unstable.git;a=summary qemu-xen-unstable.git], [http://xenbits.xen.org/gitweb/?p=qemu-xen-4.3-testing.git;a=summary qemu-xen-4.3-testing.git] etc.
  +
  +
These trees are still around for reference, but are not in use any more.
  +
  +
= Why is <code>qemu-system-i386</code> used even on x86_64 and even non-x86? =
  +
  +
QEMU in a Xen system only provides device model (DM) emulation and not any CPU instruction emulation, so the nominal arch doesn't actually matter and Xen builds i386 everywhere as a basically arbitrary choice.
  +
  +
It happens that the Xen DM part of QEMU is quite closely tied to the x86 scaffolding for various historical reasons, so we end up using <code>qemu-system-i386</code> even e.g. on ARM!
  +
  +
There is no practical difference between <code>qemu-system-i386</code> and <code>qemu-system-x86_64</code>, they should be interchangeable. However only <code>qemu-system-i386</code> is regularly tested by Xen Project (via osstest).
  +
  +
= Using a distribution supplied version of Qemu =
  +
  +
By default Xen will build its own copy of the upstream qemu tree from a branch hosted on [http://xenbits.xen.org xenbits]. However it is possible to use Xen with the qemu supplied by the distribution, assuming it is new enough and is compiled with Xen support.
  +
  +
You can configure this at build time by using:
  +
  +
./configure --with-system-qemu=/path/some/qemu-bin/
  +
  +
This will cause the Xen toolstack to use binary when launching qemu. If you specify the 'with-system-qemu' option but do not specify a path then the default will be to call <tt>qemu-system-i386</tt> using the system's default search path.
  +
  +
Note that Xen does not use Qemu for processor emulation and therefore makes no distinction between <tt>qemu-system-i386</tt> and <tt>qemu-system-x86_64</tt>, both of which can be used with either 32- or 64-bit guests. By default Xen uses <code>qemu-system-i386</code>.
  +
  +
As well as building Xen with support for the system version of Qemu it is also possible to override the binary to be used via ''device_model_override'' field in your xl vm configuration (other toolstacks may or may not expose this option). For example:
  +
  +
device_model_override = "/usr/bin/qemu-system-x86_64"
  +
  +
Note that it is you responsibility to ensure that 'device_model_version' is set to either 'qemu-xen-traditional' or 'qemu-xen' as appropriate when overriding the binary to use.
  +
  +
= Xen build system integration =
  +
  +
By default building Xen will cause a suitable version of Qemu to be downloaded and built.
  +
  +
== When xen builds qemu ==
  +
  +
[http://xenbits.xen.org/gitweb/?p=xen.git xen] will build qemu by default if it is supported on your target.
  +
  +
== Where Xen builds qemu ==
  +
  +
Qemu will build it under the <code>tools/qemu-xen-dir</code> subdirectory of your xen source tree.
  +
  +
== How xen builds qemu ==
  +
  +
When running ''make'' when building xen in the [http://xenbits.xen.org/gitweb/?p=xen.git xen git tree] the enabled qemu targets will be built and by default on x86 and x86_64 two targets are built:
  +
  +
* CONFIG_QEMU_TRAD: checks out the git tree defined by CONFIG_QEMU
  +
git://xenbits.xen.org/qemu-xen-unstable.git
  +
* CONFIG_QEMU_XEN: checks out the git tree defined by QEMU_UPSTREAM_URL
  +
git://xenbits.xen.org/qemu-upstream-unstable.git
  +
  +
== Overriding QEMU URL and revision ==
  +
  +
If you'd like to modify the URL so that the another upstream version of qemu is used, and override the branch used you can edit your "xen/.Config.mk" file, for example:
   
 
QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
 
QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
 
QEMU_UPSTREAM_REVISION = master
 
QEMU_UPSTREAM_REVISION = master
   
  +
== Building your own qemu ==
   
  +
* Get Xen source from the [http://xenbits.xen.org/gitweb/?p=xen.git xen git tree]
Then, add "device_model_version = 'qemu-xen'" in your vm config file to use this new qemu.
 
  +
* Get QEMU upstream source from: [git://git.qemu.org/qemu.git qemu git tree]
  +
* Build Xen:
   
Or build qemu out of the xen-unstable dir:
+
make xen tools
   
  +
If it fail see below.
== Get the source ==
 
  +
=== Build QEMU upstream with Xen ===
* Get Xen source (version 4.2 aka xen-unstable development version)
 
* Get QEMU upstream source from:
 
** git://git.qemu.org/qemu.git
 
   
  +
./configure --enable-xen --target-list=i386-softmmu \
 
  +
--extra-cflags="-I$path_to_xen_source/tools/include -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" \
== Build it! ==
 
  +
--extra-ldflags="-L$path_to_xen_source/tools/libxc -L$path_to_xen_source/tools/xenstore"
* Build Xen
 
  +
make
** make xen tools, if it fail see bellow.
 
* Build QEMU upstream with Xen
 
** ./configure --enable-xen --target-list=i386-softmmu --extra-cflags="-I$path_to_xen_source/tools/include -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" --extra-ldflags="-L$path_to_xen_source/tools/libxc -L$path_to_xen_source/tools/xenstore"
 
** make
 
   
 
Troubleshooting compilation errors: If you get an error from configure like "ERROR: User requested feature xen ERROR: configure was not able to find it" then see this: http://xen.1045712.n5.nabble.com/Upstream-Qemu-With-Xen-configuration-problem-td4561779.html , ie. you need to point the paths for "configure" to xen source directory, not to xen dist directory.
 
Troubleshooting compilation errors: If you get an error from configure like "ERROR: User requested feature xen ERROR: configure was not able to find it" then see this: http://xen.1045712.n5.nabble.com/Upstream-Qemu-With-Xen-configuration-problem-td4561779.html , ie. you need to point the paths for "configure" to xen source directory, not to xen dist directory.
   
  +
==== Other QEMU configure options ====
== Run it! ==
 
  +
Some features of QEMU that a Xen guest might use may not be built if a dependency isn't installed. Here is a list of <code>./configure</code> options to use:
Either:
 
  +
* Xen 9pfs
* With the latest xen-unstable tree, use:
 
  +
--enable-virtfs
** device_model_version = 'qemu-xen'
 
  +
* PVUSB
** device_model_override = '/root/work/qemu/_build/i386-softmmu/qemu-system-i386'
 
  +
--enable-libusb
'''Now xl will always try run qemu upstream with seabios, so build xen with seabios.'''
 
* For troubleshooting device_model on older Xen related errors (Xen specific options -domain-name, -vcpus, etc) see: http://xen.1045712.n5.nabble.com/Upstream-QEMU-not-working-with-Xen-unstable-lacks-option-definitions-td4372252.html
 
   
== Use with SeaBIOS ==
+
= Use with SeaBIOS =
   
SeaBIOS support is now fully integrated into the Xen build system (in xen-unstable and xen-4.2 onwards) and is always used when using ''device_model_version = "qemu-xen"''.
+
SeaBIOS support is now fully integrated into the Xen build system and is always used when using ''device_model_version = "qemu-xen"''.
   
 
If you want to build a version of SeaBIOS other than the [http://xenbits.xen.org/gitweb/?p=seabios.git;a=shortlog;h=refs/heads/xen-unstable default] then you can override ''SEABIOS_UPSTREAM_URL'' and/or ''SEABIOS_UPSTREAM_TAG'' via ''.config''. For reference the upstream SeaBIOS repository is [[git://git.seabios.org/seabios.git]].
 
If you want to build a version of SeaBIOS other than the [http://xenbits.xen.org/gitweb/?p=seabios.git;a=shortlog;h=refs/heads/xen-unstable default] then you can override ''SEABIOS_UPSTREAM_URL'' and/or ''SEABIOS_UPSTREAM_TAG'' via ''.config''. For reference the upstream SeaBIOS repository is [[git://git.seabios.org/seabios.git]].
Line 53: Line 122:
   
 
== SPICE / QXL ==
 
== SPICE / QXL ==
SPICE is an other remote-display protocol and QXL is a PV framebuffer which use the best of the SPICE capability.
+
SPICE is another remote-display protocol and QXL is a PV framebuffer which uses the best of the SPICE capabilities.
 
To activate SPICE you can add this in the VM config file (this works only with xl).
 
To activate SPICE you can add this in the VM config file (this works only with xl).
 
spice=1
 
spice=1
Line 60: Line 129:
 
spicedisable_ticketing=1
 
spicedisable_ticketing=1
   
  +
With xen 4.4 usb redirection, vdagent and clipboard sharing was added:
If you which to use QXL as well you'll have to add:
 
  +
device_model_args_hvm = [ '-vga', 'qxl' ]
 
  +
- for use usb redirection add in domU's xl cfg spiceusbredirection=N where N is number of channels one for each usb device that can be redirected, max 4.
videoram = 128
 
   
  +
- for use spice vdagent add the line below in domU's xl cfg:
On recent xen-unstable qxl support is included in libxl, you need to add only vga="qxl".
 
  +
spicevdagent=1
  +
- for use spice clipboard sharing
  +
spice_clipboard_sharing=1
   
  +
QXL currently work under Xen only on windows but xl patch is available for testing.
Actually QXL under xen works only as standard vga, for full qxl working other changes/fixes are probably needed on hvmloader.
 
  +
(check [[http://wiki.xen.org/wiki/SPICE_support_in_Xen SPICE page]] for more information.)
   
 
= Difference with qemu-xen-traditional =
 
= Difference with qemu-xen-traditional =
 
== Missing feature from the good old qemu-dm ==
 
== Missing feature from the good old qemu-dm ==
 
Currently, there is few missing feature that not supported by QEMU upstream, but the work is ongoing.
 
Currently, there is few missing feature that not supported by QEMU upstream, but the work is ongoing.
* '''dirty log''': This is needed to migrate a guest. Patches have been posted to both QEMU and libxl, but are not applied yet.
 
 
* '''VGA passthrough''': Sometime (all the time?), simple PCI passthrough does not works for graphics card. The few things that are done in qemu-dm are not yet upstreamed.
 
* '''VGA passthrough''': Sometime (all the time?), simple PCI passthrough does not works for graphics card. The few things that are done in qemu-dm are not yet upstreamed.
   
 
== New feature from QEMU upstream ==
 
== New feature from QEMU upstream ==
There is some benefice of upstreaming the support of Xen to QEMU, we can now use:
+
There are some benefits from upstreaming the support of Xen to QEMU.
  +
We can now use:
* VirtIO, as an alternative to Xen PV drivers
 
  +
* VirtIO as an alternative to Xen PV drivers
* SPICE, for a remote-display instead of the old VNC.
 
  +
* SPICE, as a remote-display protocol instead of the old VNC.
 
* different kind of file format for a disc.
 
* different kind of file format for a disc.
* the ability to connect several time to the VNC server.
+
* the ability to connect several times to the VNC server.
 
* ...
 
* ...

Latest revision as of 10:09, 14 February 2019

History

Historically Xen contained a fork of qemu with Xen support added, known as 'qemu-xen-traditional' in the xl toolstack. However since Qemu 1.0 support for Xen has been part of the mainline Qemu and can be used with Xen from 4.2 onwards. The xl toolstack describes this version as 'qemu-xen', and this became the default from Xen 4.3 onward.

The 'qemu-xen-traditional' fork remains available to support guest OSs that were installed using it. After a Dom0 upgrade to a newer Xen version the new 'qemu-xen' device model is recognized as a significant hardware change. Some guest operating systems, especially non-free systems that rely on licensing bound to a specific hardware, do not behave nicely when hardware is changed under their feet.

Xen Qemu Trees

The qemu-xen code is maintained upstream in the qemu.org git trees. In addition the Xen project also maintains its own stable branch of qemu, based on the upstream stable branches with a small number of additional fixes for Xen. These can be found on xenbits in qemu-xen.git with a branch for each release of Xen named 'stable-VERSION', e.g. stable-4.10.

The qemu-xen-traditional fork is maintained on xenbits in qemu-xen-traditional.git. There are branches for each release of Xen named 'stable-VERSION' e.g. stable-4.10.

By default the Xen build system will clone and build both versions of QEMU from the branches on xenbits.

Obsolete Xen Qemu Trees

In the past, we maintained the following trees:

These trees are still around for reference, but are not in use any more.

Why is qemu-system-i386 used even on x86_64 and even non-x86?

QEMU in a Xen system only provides device model (DM) emulation and not any CPU instruction emulation, so the nominal arch doesn't actually matter and Xen builds i386 everywhere as a basically arbitrary choice.

It happens that the Xen DM part of QEMU is quite closely tied to the x86 scaffolding for various historical reasons, so we end up using qemu-system-i386 even e.g. on ARM!

There is no practical difference between qemu-system-i386 and qemu-system-x86_64, they should be interchangeable. However only qemu-system-i386 is regularly tested by Xen Project (via osstest).

Using a distribution supplied version of Qemu

By default Xen will build its own copy of the upstream qemu tree from a branch hosted on xenbits. However it is possible to use Xen with the qemu supplied by the distribution, assuming it is new enough and is compiled with Xen support.

You can configure this at build time by using:

   ./configure --with-system-qemu=/path/some/qemu-bin/

This will cause the Xen toolstack to use binary when launching qemu. If you specify the 'with-system-qemu' option but do not specify a path then the default will be to call qemu-system-i386 using the system's default search path.

Note that Xen does not use Qemu for processor emulation and therefore makes no distinction between qemu-system-i386 and qemu-system-x86_64, both of which can be used with either 32- or 64-bit guests. By default Xen uses qemu-system-i386.

As well as building Xen with support for the system version of Qemu it is also possible to override the binary to be used via device_model_override field in your xl vm configuration (other toolstacks may or may not expose this option). For example:

   device_model_override = "/usr/bin/qemu-system-x86_64"

Note that it is you responsibility to ensure that 'device_model_version' is set to either 'qemu-xen-traditional' or 'qemu-xen' as appropriate when overriding the binary to use.

Xen build system integration

By default building Xen will cause a suitable version of Qemu to be downloaded and built.

When xen builds qemu

xen will build qemu by default if it is supported on your target.

Where Xen builds qemu

Qemu will build it under the tools/qemu-xen-dir subdirectory of your xen source tree.

How xen builds qemu

When running make when building xen in the xen git tree the enabled qemu targets will be built and by default on x86 and x86_64 two targets are built:

 * CONFIG_QEMU_TRAD: checks out the git tree defined by CONFIG_QEMU
   git://xenbits.xen.org/qemu-xen-unstable.git
 * CONFIG_QEMU_XEN:  checks out the git tree defined by QEMU_UPSTREAM_URL
   git://xenbits.xen.org/qemu-upstream-unstable.git

Overriding QEMU URL and revision

If you'd like to modify the URL so that the another upstream version of qemu is used, and override the branch used you can edit your "xen/.Config.mk" file, for example:

 QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
 QEMU_UPSTREAM_REVISION = master

Building your own qemu

   make xen tools

If it fail see below.

Build QEMU upstream with Xen

   ./configure --enable-xen --target-list=i386-softmmu \
       --extra-cflags="-I$path_to_xen_source/tools/include -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" \
       --extra-ldflags="-L$path_to_xen_source/tools/libxc -L$path_to_xen_source/tools/xenstore"
   make

Troubleshooting compilation errors: If you get an error from configure like "ERROR: User requested feature xen ERROR: configure was not able to find it" then see this: http://xen.1045712.n5.nabble.com/Upstream-Qemu-With-Xen-configuration-problem-td4561779.html , ie. you need to point the paths for "configure" to xen source directory, not to xen dist directory.

Other QEMU configure options

Some features of QEMU that a Xen guest might use may not be built if a dependency isn't installed. Here is a list of ./configure options to use:

  • Xen 9pfs
 --enable-virtfs
  • PVUSB
 --enable-libusb

Use with SeaBIOS

SeaBIOS support is now fully integrated into the Xen build system and is always used when using device_model_version = "qemu-xen".

If you want to build a version of SeaBIOS other than the default then you can override SEABIOS_UPSTREAM_URL and/or SEABIOS_UPSTREAM_TAG via .config. For reference the upstream SeaBIOS repository is [[1]].

New features

VirtIO

It's the PV drivers from KVM world. To use it for network, just use 'virtio-net' as a model:

 vif = [ 'model=virtio-net' ]

You will need those kernel modules to use a VirtIO device

 virtio_mmio virtio_pci virtio_net

SPICE / QXL

SPICE is another remote-display protocol and QXL is a PV framebuffer which uses the best of the SPICE capabilities. To activate SPICE you can add this in the VM config file (this works only with xl).

 spice=1
 spicehost='0.0.0.0'
 spiceport=6000
 spicedisable_ticketing=1

With xen 4.4 usb redirection, vdagent and clipboard sharing was added:

- for use usb redirection add in domU's xl cfg spiceusbredirection=N where N is number of channels one for each usb device that can be redirected, max 4.

- for use spice vdagent add the line below in domU's xl cfg:

 spicevdagent=1

- for use spice clipboard sharing

 spice_clipboard_sharing=1

QXL currently work under Xen only on windows but xl patch is available for testing. (check [SPICE page] for more information.)

Difference with qemu-xen-traditional

Missing feature from the good old qemu-dm

Currently, there is few missing feature that not supported by QEMU upstream, but the work is ongoing.

  • VGA passthrough: Sometime (all the time?), simple PCI passthrough does not works for graphics card. The few things that are done in qemu-dm are not yet upstreamed.

New feature from QEMU upstream

There are some benefits from upstreaming the support of Xen to QEMU. We can now use:

  • VirtIO as an alternative to Xen PV drivers
  • SPICE, as a remote-display protocol instead of the old VNC.
  • different kind of file format for a disc.
  • the ability to connect several times to the VNC server.
  • ...