Xen Document Days/TODO

The Xen Project Wiki has been subject to a severe spam attack in the last several months. To solve this and keep the wiki usable for everyone, we had to lock down the wiki and create an editors group. Please fill out this form and we will add you to the group.

Go back to Xen Project Document Days

Requested Documentation

Feel free to comment on items using the following convention:

* {{Comment|~~~~:}} your comment

Feel free to add new workitems using the following convention:

==== Workitem ====
~~~~: work-item description

THEME OF THE MONTH: "Ready for 4.6"

This month, we prepare for the release of Xen Project 4.6 early next month. We need to make sure that users of the new release can find the documentation they need to make it all work. So, this month, we need to:

Add new documentation for new features
Modify existing documentation for anything which is changing in the newest release, and
Deprecate old documentation, letting people know that certain information is applies only to older releases

Check out the current documentation, and anything which doesn't map to 4.6 (or 4.5) commands or best practices will need improvement.

Items Requested by Users

The following requests have appeared in our IRC channel and UserVoice for additional documentation. Recent requests include the following (feel free to add additional topics as they appear):

Development Documentation: An overview of the driver architecture. This may already exist, but it needs to be more prominently visible. If it's out there, it is not as easy to find as it should be.
User Documentation: Simple example of moving a VM from one machine to another offline and possibly changing toolsets as it goes. Wei recently gave the summary below in ##xen. We need a page with more meat on the bones, particularly for migrating between toolkits and across multiple versions of the hypervisor (e.g., moving from 4.1 to 4.4).

xm save domain image; scp image user@remote:~; scp domu.cfg user@remote:~; 
modify domu.cfg on remote host; xl restore

User Documentation: Update to the Xen_Windows_GplPv doc. The drivers are still used, but the information on the page is dated. It needs a refresh.
Build a NEW HOW-TO describing how to create a Xen 4.5.1 AMD64 environment. This should include sub-HOWTOs for networking and DomU creation.

From Uservoice:
All the examples for installing and using Xen are currently antiquated.
Can someone just record how they set up a new system using 
Ubuntu 14/15, Xen 4.5.1, and XAPI/xl? Sections on setting up 
networking, passthru, BIOS bugs and *NIX/Win/OSX would be 
exactly what is needed.

New HowTos Needed

We've got a lot of good information in the documentation, but it isn't always in a form that will help someone use a particular feature for the first time. Or, the information is present, but it might need to be brought up-to-date.

Some of these possible topics include:

Installing on ARM (need a basic HowTo)
Using Xen Project with common applications (see this Q&A post about Citrix Netscaler)
Get up and running with MirageOS
HowTo use smbios_firmware option. See this thread for background.
Any other niche topic which doesn't have a good HowTo-type doc page

Man pages loose ends

move the xenstat/xentrace manpages into docs
there are a few misc documents scattered through tools/ which would be better moved into docs/

THE GREAT 4.5 CLEAN-UP

There are a lot of pages which rely on xm which need to be updated to xl. If you notice one and don't have time to fix it, add it here for others to do:

Xen_Project_Beginners_Guide (should default to xl, and give a nod to xm for old releases)
- Lars.kurth 12:00, 27 February 2015 (UTC) : This may not be entirely straightforward and may need to wait until we have a Debian version shipping Xen 4.5 - although jessie ships 4.4. Some of the text also refers to quite old xen versions, which should be looked at. For example it does use xen-tools and other external tools are in use and someone will need to go through the page and check whether everything works as intended.
- For example xen-tools, xt-create-xen-config seems to have something which writes a sudo stanza for you, involving xm. xm-create-image has the ability to start images with xm. It does NOT have a config option (--xen-toolstack=xl) for changing the command, but contains a heuristic to try and figure our what to use (see [1]). Thus, it may be easy to fix, although AFAICT it isn't used everywhere. Information from "dpkg -L xen-tools | xargs grep -e '\bxm\b' |less". Overall conclusion: there are some nits to pick, but by and large it will work just fine with xl. The package ought to be updated.

Performance-related Documentation

The documentation on Performance Tuning seems thin, given the importance of the subject:

Category:Performance Lists the Performance docs currently available. It seems rather sparse.
Network_Throughput_and_Performance_Guide Has lots of interesting information, but it is oriented for XenServer/XCP. This needs to be recast as a true Xen Project document.
Tuning_Xen_for_Performance Is potentially the most important document, but it has only a few topics and not that many details. There is some discussion about this page in section 3.12

Integration Documentation

These documents are for projects which users need to use in conjunction with Xen Project. The goal of these documents is not to repeat the documentation found in other projects, but to give people the essential information required to integrate them into their Xen Project installation.

Most of these documents are still quite new and can benefit from improvements by people who have used these projects.

Using Ceph storage with Xen Project
Using Gluster file system with Xen Project
Using ConVirt virtualization management platform with Xen Project
Using Xen Project in OpenStack clouds
Using Xen Project in CloudStack clouds
Using Xen Project in OpenNebula clouds
Unikernels (e.g., Mirage OS, LING, HaLVM, OSv, etc.) and how they relate to Xen Project

Missing bits in Beginners Guide

Xen_Project_Beginners_Guide has a few TODOs in it. In particular, it is missing some diagrams

(IMPORTANT) Linux Distros

It seems that the list of Linux Distros which support Xen is a lot larger than listed on this wiki.

Easy tasks for Beginners

You may want to read Wiki Help if you do not have an account.

Migrate useful blog posts to wiki

Sift through blog.xenproject.org and copy useful posts to wiki. This will mainly be a reformatting exercise. Before you copy an article, perform a search on the wiki (headline or key word) to see whether the post exists already. If in doubt, ask on the IRC channel.

Fix articles that need attention

Category:Contains_Needs_Formatting - Pages that need formatting
Category:Contains_Needs_Action - Pages that need fixing. Go through these, and if you get stuck ask people on the IRC channel for help, or go to the next page.
Category:Contains_TODO - Pages that have a TODO item on the page. Go through these, and if you get stuck ask people on the IRC channel for help, or go to the next page.

Review Categorization

Sometimes new pages on the wiki are not created with categories. You can find these by going to Special:UncategorizedPages. Documentation about which category is used where can be found in Categories for Authors. If in doubt ask on IRC.

Pages With Comments or Improvement Suggestions

We ask users to highlight issues using talk pages, following the conventions outlined in Help:Talk Pages. Comments made using these conventions can be identified using the Category:Pages with comments.

Links to Developer Documents on XenBits

There are a number of developer-related documents which are hosted on XenBits which should be locatable through this Wiki. Ideally, someone should work backwards from the XenBits document index to make sure each document exists in a link from the appropriate Wiki page.

If no appropriate Wiki page exists, one should be created or listed here for creation. For function-specific docs which don't necessarily deserve a Wiki page, we should have a catch-all page to let people know that these docs exist.

Missing Pages

Special:WantedPages - You can find missing pages by going to this link. There are several courses of action that can be taken:
- The page is needed : Add {{TODO|With a comment}} to the top of the page
- The page is not needed : Remove the reference to the page (follow the link in the bracket besides the page)
- Note that the list of Wanted pages contains lots of needed translations and categories, which you can ignore
- If in doubt, ask for help on the IRC channel

Double-p 09:55, 25 February 2013 (UTC): There are a gazillion Template-Whatever including a TODO, really needed? Lars.kurth 17:40, 25 March 2013 (UTC): See #Special:WantedPages_broken

Update links to defunct gentoo-wiki.com site

According to [2] it is lost and not coming back. That site has archives but it seems like [3] is the new home. Pages referring to gentoo-wiki.info ought to be updated.

Xen : Open work items

Review and Update: xenpm usage recommendations

There is Xen power management and Xenpm command which may all be outdated. These should probably be reviewed and updated.

Lars.kurth 10:09, 12 February 2015 (UTC): There has been a bit of activity on Talk:Xen power management which needs to be merged

User focused guide for setting up host EFI boot

We have http://xenbits.xenproject.org/docs/unstable/misc/efi.html but could use something a bit more user focused, e.g. a walkthrough type guide.

UEFI Secure Boot

Write a document on how to sign the xen binary for UEFI Secure Boot. Potential reference here: Dealing With Secure Boot

Mirage OS

While much documentation is available at their website, we need to have pages here which describe Mirage OS and give useful pointers to their docs.

Ideally, it would be good to have extensive documentation for this project on this Wiki, as Mirage OS is being fostered by Xen Project.

Documentation on lib(xen)vchan

The header has some stuff, but a more complete guide would be useful.

The usage of the example code is also not documented. http://lists.xen.org/archives/html/xen-users/2014-01/msg00162.html has some (incomplete) info.

Update Xen Windows GplPv page

The page is sorely out of date (e.g., the download pages don't appear to exist anymore). Even though there are other Windows PV drivers, this set is still in demand.

Describe the different ways, and necessary components, to create PVs using xl

Raised by mail by Matthias Blankenhaus (who is prepared to take the lead, but needs help): This document would describe the different ways, and necessary components that each way requires, to create PVs using xl. Right now there are at least the following cases:

loopbacked vhds
- single partition format
- disk format
qemu backed vhds
- qcow / qcow2
LVS backed image

This is not a complete list. Also the mileage of each case differs depending on the kernel features enabled in dom0/domu and also depending on the Xen version. Furthermore, certain format require additional infrastructure like the tap device.

All in all, this a complex topic that needs clarification. Doing this based on the lowest common denominator, xl, would clarify many things. In addition, baseline performance data on all of the above would help a great deal. Matthias volunteered to run the majority of needed perf tests

HOWTO : using xl trigger to poweroff or firing off NMIs

Howto on using xl trigger to poweroff or firing off NMIs.

HOWTO: Chain pypxeboot and pygrub

How to chain pypxeboot and pygrub

Lars.kurth 10:35, 27 March 2012 (UTC): Is pypxeboot popular at all?
Fheigl: I added that. It is somewhat common, OVM 2 and OVM 3 rely on it for PXE installs. I'm not sure if xenpvnetboot is the same, or works. The use case is "you want to use the same installation server for Xen VMs and phys. servers. You want to be able to reinstall them based on PXE reply.
Fheigl: There is documentation for pypxeboot, but there is no (afaik) documentation for how pygrub is used as a fallback loader. We wasted many days just to get some hack that can't even load a newer kernel. Which sucks, in the end you spend more time/money making PV PXE boot work than you save by using virtualized servers.
Fheigl: How could it be popular given the state of documentation. <= Key issue to the left of this sentence.
Lars.kurth 16:03, 19 September 2012 (UTC): wondering whether this video contains what is needed
Ijc 11:01, 25 June 2014 (UTC): Although I've not watched the whole thing I stepped through that video enough to see all the slides (I think) and I didn't see anything about this topic or anything like it.
Ijc 11:17, 25 June 2014 (UTC): I've downloaded pypxeboot 0.0.3 and as far as I can tell the fallback to pygrub is supposed to happen automatically if pxelinyx.cfg says to do a local boot. If this doesn't Just Work then I think that is a code bug rather than a documentation issue. I'd recommend retiring this docs TODO in favour of a bug report to the pypxeboot authors and/or xen-devel. I googled around a bit and didn't find any existing bug reports anywhere.

Refactor/refresh ARM h/w information

(DONE) Xen ARM with Virtualization Extensions effectively duplicates the list of supported h/w in #Hardware and #Platform specific configuration, make it a single list.
Update Arndale info to be less Linaro specific now that mainline is usable.
Refactor Xen ARM with Virtualization Extensions/OdroidXU and Xen ARM with Virtualization Extensions/Arndale into a generic Eynos page + board specific notes. (and check other platform vs. board combinations).

Visually distinguish Xen, MirageOS and Community Documentation better

Lars.kurth 15:32, 28 January 2013 (UTC): Users are getting confused when looking at wiki pages. If there was a visual way to distinguish Xen, MirageOS and Other content better, this would help. We could use colors and skinning when it comes to various elements, as well as mascots/logos:

For example: Help:Contents has a different feel (and colour scheme)
This could be replicated in trail boxes and other elements
We can also use mascots

But first, I need to investigate what is possible.

Lars.kurth 16:14, 28 January 2013 (UTC): Looks as if the only way this can be done is via Custom Namespaces. The only way to do this properly is to

Have different namespaces for Xen, MirageOS, ... and Community stuff and keep all the common stuff in Xen
The name spaces could then be styled differently, for example the Help namespace includes the ns-12 style into the body attributes, whereas the Main (default) namespace uses the ns-14 style.
Custom namespaces can be created by changing the config and are linked to custom styles
The styles are then defined in MediaWiki:Common.css

Implications of using custom namespaces:

Search: Pages in a custom namespace do not show up in default search results. So if we had a custom MirageOS namespace, it would mean that search results for MirageOS would not show up. One would need to go to Advanced Search and select MirageOS
Rename: Renaming pages is possible and as straightforward as renaming pages
Xen Namespace: The Xen namespace exists already, this may need to be renamed
Categories and Namespaces: Categories cross namespaces. In other words, Category:XXX would go across all namespaces. As an example, see Category:Wiki_Management

Mediwiki Issues

Special:WantedPages broken

Issue 1: Pages that need translation are spamming the list

Solution: fix language templates and mark with {{Stub translation}}
Lars.kurth 16:48, 28 January 2013 (UTC): There is something wrong with the language templates, essentially spamming the wanted pages. Do ignore these instructions until fixed.
Lars.kurth 23:28, 11 February 2013 (UTC): Fixed language template issues by removing most languages from the templates
Lars.kurth 18:03, 25 March 2013 (UTC): Started marking pages with {{Stub translation}}

Issue 2: Missing templates are spamming the list

Solution: mark missing templates with {{Undefined Template}}
Lars.kurth 17:27, 25 March 2013 (UTC): Started marking all missing templates with {{Undefined Template}}

Issue 3: Missing categories are spamming the list

Solution: mark with {{Stub category}}
Lars.kurth 18:03, 25 March 2013 (UTC): Started marking pages with {{Stub category}}

Categories & Indexes

Lars.kurth 13:54, 29 January 2013 (UTC): Combined categories such as list all XCP FAQs have been asked for sevceral times
Investigate plug-ins

Developer Docs

Docs that need attention or are missing

Make XAPI docs (built from source) available on XenProject.org
some more xenstore docs (i.e. recent python bindings that were sent to the -users list)
- Ijc 12:13, 25 June 2014 (UTC): I don't think these are included in Xen? Need a reference.
Using libxenstat python bindings (imo they're broken)
- Ijc 12:13, 25 June 2014 (UTC): Is this a bug report? If they are broken we should fix them (as well as potentially documenting them)
More PV protocol docs (see recent patches to blkif.h for example)
- Ijc 12:13, 25 June 2014 (UTC): blkif.h and netif.h are perhaps the most critical ones and those are now pretty good. But the rest (fbif.h, kbdif.h, pciif.h, usbif.h, tmpif.h and vscsiif.h) could all stand to be improved.
Document xenstore paths used by guests and toolstack etc
- Ijc 16:10, 26 November 2012 (UTC): This is now in tree: http://xenbits.xen.org/docs/unstable/misc/xenstore-paths.html

Done Recently

In progress (some just need double checking whether finished)

FreeBSD_Dom0 - confirmed up-to-date (Feb, 2015)

Draft a list of Questions for a User Survey

Lars.kurth 10:09, 30 April 2014 (UTC) : See Xen Project User Survey
Lars.kurth 4 Feb 2015 (UTC) : Russell and Sarah are driving this now

Xen 4.3 / 4.4 Documentation : Spice Config Example for upstream QEMU

Take this thread and convert into an Example

Lars.kurth 16:56, 9 April 2014 (UTC) Created SPICE_support_in_Xen for now