Asking Xen User Questions

Needs Refactor Needs splitting and moving into the smaller FAQs on Category:FAQ

Xen Users Mailing List Commonly Asked Questions

Introduction

This document is a Xen.org community effort to gather the most commonly asked questions from the xen-users emailing list and other support tools to assist new and experienced Xen hypervisor users with problems that frequently arise. If you would like to add content to this document, please send an email to community.manager@xen.org for editing rights if you don't have wiki editing rights already. For those users interested in trying Xen without installing the application, a Live CD version is available at http://wiki.xensource.com/xenwiki/LiveCD.

Support Tools

The following sites are available for Xen hypervisor support:

• xen-users mailing list - http://lists.xensource.com/mailman/listinfo/xen-users
• Xen 3.3 Documentation - http://bits.xensource.com/Xen/docs/user.pdf
• Stack Oveflow - http://stackoverflow.com/
• Complete email history of all xen mailing lists - http://xen.markmail.org
• Language Specific Support
  • Japanese - http://lists.xensource.com/mailman/listinfo/xen-japanese
  • Portuguese - http://groups.google.com/group/xen-br

How To Guide Links

The Xen.org community Wiki has a HowTo Page with various information sources at http://wiki.xensource.com/xenwiki/HowTos. Updates to this Wiki page are continuous so check back often for new How Tos. Sample topics internally within the Wiki are:

• Xen Networking
• Xen FreeBSD

Sample topics with external links are:

• Adding USB Devices to Xen HVM - http://www.virtuatopia.com/index.php
• Xen on Fedora - http://rackerhacker.com/2011/08/05/xen-4-1-on-fedora-15-with-linux-3-0/
• Create and Install CentOS on Xen - http://wiki.centos.org/HowTos/Xen/InstallingCentOSDomU
• http://www.virtuatopia.com/index.php/Configuring_a_VNC_based_Graphical_Console_for_a_Xen_Paravirtualized_domainU_Guest
• https://help.ubuntu.com/community/Xen Xen on Ubuntu

Guest Related Questions

Guest Conversion

Q (G1.0): How do I convert a Centos HVM Guest to a PV Guest?

A (G1.0): Creating a Centos HVM domU with working PV drivers : http://pastebin.com/fb6fe631 Converting HVM guest to PV guest : http://pastebin.com/f6a5022bf If you follow both parts correctly you should have a working PV domU. If anything goes wrong during conversion process, you should still be able to boot the previous HVM domU config if you select the non-xen kernel (second entry) from grub menu.list.

Console

Q (G2.0): I have an Xen image that was built for a graphical console (VNC). Is there any way to change it to the non-graphical console (xen console)?

A (G2.0): For HVM guest, you need to enable serial port on domU config file (example here: http://pastebin.com/fb6fe631), and setup domU to use serial port (ttyS0 on Linux) by modifying (for Linux domU) /boot/grub/menu.lst, /etc/inittab, and /etc/securetty. If it's PV guest, you need to set up domU to use xen console (which is xvc0 on current xen version, hvc0 on pv_ops kernel). It's similar to setting up domU for serial console, you just need to change ttyS0 to hvc0. An example of domU setup that can use both xvc0 and vnc console is here : http://pastebin.com/f6a5022bf

Q (G2.1): How do I remove an active virtual machine?

A (G2.1): xl shutdown or xl delete

Q (G2.2): How do I run xl console to a WindowsXP DomU?

A (G2.2): You can't xl console to that (I'm not sure you can xl console to any hvm, but I know you can't to one that doesn't have a console).

Q (G2.3): I start a new DomainU (Guest) and some text scrolls by for launching the guest but then it just sits there with Continue and no actions takes place?

A (G2.3): The console for this new DomainU is not properly available for you; fix this by adding xtra="xencons=tty" in the configuration file. This will bring up a login screen directly for your new DomainU.

Q (G2.4): One of our CentOS 5.3 randomly reboots, at different times of the day, and I can't see why it's doing it. I have looked through the logs, but don't see any thing in there that shows me why it has rebooted. How can I debug this?

A (G2.4): The problem is that when the box panics, it stops syslogd, so you don't get the panic output in /var/log. The best way to fix this is to setup a logging serial console.

Q (G2.5): Hi, i want to stop VMs, but when i execute xl destroy vmname the VM disappears completly from mu vm list (xl list) How can I stop them without delete them?

A (G2.5): You've probably been using "xl create". That's the way it works.

Drivers

See Xen_FAQ_Drivers,_Windows

Domain0

See Xen_FAQ_Dom0

DomainU

Q (G5.0): My DomU does not fully start; it shows the following output stopping at Continue...

$ sudo xm console test

io scheduler cfq registered

RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize

Xen virtual console successfully installed as xvc0 Event-channel device installed. netfront: Initialising virtual ethernet driver. i8042.c: No controller found. mice: PS/2 mouse device common for all mice TCP bic registered NET: Registered protocol family 1

NET: Registered protocol family 17

Using IPI No-Shortcut mode xen-vbd: registered block device major 8 blkfront: sda2: barriers enabled XENBUS: Device with no driver: device/console/0 Freeing unused kernel memory: 140k freed kjournald starting. Commit interval 5 seconds

EXT3-fs: mounted filesystem with ordered data mode.

• **************************************************************
• **************************************************************
• * WARNING: Currently emulating unsupported memory accesses **
• * in /lib/tls glibc libraries. The emulation is **
• * slow. To ensure full performance you should **
• * install a 'xen-friendly' (nosegneg) version of **
• * the library, or disable tls support by executing **
• * the following as root: **
• * mv /lib/tls /lib/tls.disabled **
• * Offending process: modprobe (pid=663) **
• **************************************************************
• **************************************************************

Continuing...

A (G5.0): It might only be that you don't have a VPS physical console, and that your VPS is fully booted, but you can't see it. There are few things to check.

First, check that your VPS has a "console" device in /dev. Mount your domU filesystem in the dom0, go in /dev and do:

/dev/MAKEDEV console

If you are using a modern Xen kernel and hypervisor, you should check the parameters of the startup file. Check that it has the following option:

extra = "4 TERM=xterm xencons=tty console=tty1"

Then start your VPS and watch it booting. Note that once it's booted up, you should check that it has a xen friendly libc6 installed (in Debian, you would do "apt-get install libc6-xen").

Q (G5.1): is there a way to set the credit-scheduler's limits and weights per domU in the domU configuration file?

A (G5.1): weight= in the xm config file works, unless you are using RHEL or CentOS.

Q (G5.2): How do I start an application from within a DomU?

A (G5.2): Well, you could always just log in to that VM, open a terminal and run the program. Or you could SSH or telnet in to the VM, start a screen session and run the program. A VM acts just like any other server, so the proceedure for starting programs and executing commands locally and remotely are exactly the same as doing so on any computer.

Q (G5.3): My domUs are in a permanent 'b' (blocked) status as shown by 'xm list', even though they are functioning just fine. That's not normal, is it?

A (G5.3): It's normal for them to show as blocked when they aren't actively running something - in the same way that any process on a 'normal' machine will show as blocked when it's waiting for input, each guest will show as blocked when it's got nothing to do. Give something a processor intensive task to do and you'll find it changes state to running (at least some of the time).

Q (G5.4): Is there any way, to get the name of a domU from the network-common script?

A (G5.4): hostname=$(xenstore_read "$XENBUS_PATH/domain" | tr -- '_.:/+' '-----')

Q (G5.5): Is it possible to increase the screen resolution of my xen guest Windows Vista?

A (G5.5): On current Xen, with stdvga=1 & videoram=16, resolutions up to 2048x1536x32 are possible. All that said, the RDP suggestion is probably a better way to access the guest in any case.

Q (G5.6): How to install Solaris via HTTP as a para guest?

A (G5.6): Solaris 10 can only be used as HVM guest. OpenSolaris can be used as PV guest, installed from iso. You can't install it from http. Once you have it installed, you also need zfs support for pygrub (either that, or manually copying kernel and boot archive to dom0)

Boris provides some nice examples on his site : http://bderzhavets.wordpress.com/

Q (G5.7): Is it possible to find out the specific vnc Display Number of a domU?

A (G5.7): virsh vncdisplay domU_name_or_id

               xenstore-ls /local/domain/domU_id/console 

Q (G5.8): I am trying to create a guest domain. I specified the configurations in /etc/xen-tools/xen-tools.conf and I ran $sudo xen-create-image --hostname=virtualrouter1 --role=udev the output is: sudo: xen-create-image: command not found

A (G5.8): Make sure you installed the Xen tools, for example: apt-get install xen-tools

Q(G5.9): I'm trying to assign a dynamic hostname to a xen instance as follows:

Cfg file kernel = "/root/vmlinuz-2.6.18-128.1.14.el5xen" ramdisk = "/root/initrd-2.6.18-128.1.14.el5xen.img" memory = 512 hostname = "uniquehostname" name = "my-vm-name"

• . . . .

In both the cases, the instance is unable to get the correct hostname..

A (G5.9): From "xm create --help_config" : hostname=NAME Set the kernel IP hostname. interface=INTF Set the kernel IP interface name. dhcp=off|dhcp Set the kernel dhcp option. On most LInux distros, kernel hostname and IP address is ignored, making it somewhat useless. You need to use your normal distro method to set hostname on domU (/etc/sysconfig/network on RHEL)

Q(G5.10): As far as I can see, there is something different between using 'xm create' and 'xm new' followed by 'xm start'. It's something to do with data being stored in XenStore. I couldn't suspend the one started with 'xm create'. Could someone please explain the effective difference between the two and when 'create' should be used instead of 'new' and vice-versa.

A(G5.10): xm create -> domU configuration is NOT managed by xend. Usually using config files on /etc/xen. This is the easiest method to use for beginners, as you have a config file that you can edit manually. The default on RHEL5 (which uses Xen 3.1+). "xm new" and "xm start" -> domU configuration is managed by xend. You change values using commands like "xm block-attach", which can modify settings online. No config file to edit manually. The default on current versions of Xen.

G (G5.11): I have problem with domU clock. It lose 30 minutes each day. How can i synchronize it with dom0 clock?

A (G5.11): Is this PV domU? If yes, setting /proc/sys/xen/independent_wallclock to 0 (the default) should make it sync with dom0. You only need ntp on dom0, and domUs will follow. The alternative, set /proc/sys/xen/independent_wallclock to 1 and run ntp on domU. If this is a HVM dom0, running ntp on domU is your friend. Also, check http://tuttodebian.blogspot.com/2008/05/xen-clocksource0-time-went-backwards.html to see if your system experience similar symptoms.

G (G5.12): I would like to set sched-cred parameters on my domU configuration file. How can i do that?

A (G5.12): cpu_cap & cpu_weight Run "xm create --help_config" for details, and read http://wiki.xensource.com/xenwiki/CreditScheduler

G (G5.13): Is it possible to increase guest memory without reboot?

A (G5.13): You can do a "xm mem-set <Domain> <memory>" for a PV domU, but you had to set maxmem higher than current assignment beforehand.

G (G5.14): Is it possible to take an already created domU sparse file and make it a non sparse file?

A (G5.14): cp --sparse=never orig.img new.img

G (G5.15): I have tried to change CD ISO images during a HVM install using the following commands but it doesn't work. After changing the CD ISO image, it doesn't detect the new ISO image. (qemu) eject -f hdc (qemu) change hdc /media/hitachi/cd-rom-image.iso

A (G5.15): Use xm block-list <domid> to find the cdrom be-path for the domain, for example: xm block-list 5 Vdev BE handle state evt-ch ring-ref BE-path 768 0 0 4 9 16383 /local/domain/0/backend/vbd/5/768 5632 0 0 1 -1 -1 /local/domain/0/backend/vbd/5/5632 Having identified the cdrom device (5632) you can check what iso image it is connected to: xenstore-read /local/domain/0/backend/vbd/5/5632/params (nothing returned) To connect a new iso image: xenstore-write /local/domain/0/backend/vbd/5/5632/params /mnt/gl3-tb1_store/MWWin2003R2SvrStdx86_BX2SVOL_EN.iso And you can now see that it is connected: xenstore-read /local/domain/0/backend/vbd/5/5632/params /mnt/gl3-tb1_store/MWWin2003R2SvrStdx86_BX2SVOL_EN.iso This method works with both emulated devices and with gplpv drivers.

Q (G5.16): Is it possible to set the xen to boot the domU one by one when server starts, as currently we have 20 domU, and if boot them together, the the hard disk will be very very slow.

A (G5.16): cd /etc/xen/config/........ && for i in * do ...... (start VM, .....)...... sleep 60 (or whatever time you think is right to start a VM) done

Q (G5.17): I use FluidVM on some of our VPS host nodes, and the management server has crashed, so now I need to recover the running VM's, somehow. FluidVM deploys the domU's on the hostnode dynamically from a database, i.e. there's no /etc/xen/vps1 (for example) config files. The domU's are still running on the servers, and I now want to create config files for them, while they're running.

How would I be able todo this?

For example, here's a list of running VM's from one of the servers:

root@usaxen02:[~]$ xm list Name ID Mem(MiB) VCPUs State Time(s) AndriesBurger_39_cronos 90 255 1 -b---- 42.4 Bruce_18_carmen 60 255 1 r----- 3528327.5 Domain-0 0 3433 4 r----- 1116681.7 Rudi_14_mars 40 3007 2 -b---- 953036.3 Rudi_44_vps2 93 255 1 -b---- 22.9

Is there any way to create a config file, /etc/xen/AndriesBurger_39_cronos, from the running domU AndriesBurger_39_cronos ?

A (G5.17):You can use "xm list -l" to dump the configuration in SXP format; then you should be able to use "xm new" or "xm create" with the "-F" option to load an SXP-based config file. See the "xm" man page for more info - that's where I dug up this.

Q (G5.18): How to set up Xen DomU as Windows 2008 Server on a CentOS Dom0 machine?

A (G5.18): Start using the normal way that you usually do when you install a HVM domU, whether it's virt-manager/virt-install or using manually-created config file. One additional thing to note is that for 64bit HVM domUs you need to make sure that acpi, apic, and pae is set to 1 on domU config file. Once you get that Win2008 fully installed, you can install GPLPV driver later to improve performance.

Q (G5.19): I need to install windows streaming media server on one of my xen3.4.2 guest. Is it possible to create a windows xp paravirtual guest on xen? Like I install other Linux guest as the paravirtual ones. If yes please give some brief steps for that.

A (G5.19): No, unless you ask Microsoft to port XP kernel to Xen PV model:)

Judging from the fact that XP was declared end of life several times, and the fact that even Hyper-V requires VT to run, I highly doubt they will ever create PV-enabled windows

Q (G5.20): I would like to move an existing W2K3 install onto my new super duper xen box -- Installing linux based machine is no issue, it's the windows ones that keep getting me. I have Acronis which we usually use for bare metal restores, and it seems that bare metal restores don't want to work too well with XEN, any assistance/help/ideas ?

A (G5.20): I have successfully migrated several physical windows 2003 installs to Xen by adding the ide drivers as per http://support.microsoft.com/kb/314082 and then using Acronis True Image to copy the disks.

One issue I had was that the boot cd created by the version of True Image we have did not boot under Xen (hung on boot screen), but Acronis support sent me a version that did work (TrueImageLinuxServer8072_multiparam_Standard_english_il.iso). However due to using emulated nic and hd running True Image directly under Xen was very slow, my first solution to that problem was to make a small windows hvm install with gplpv drivers loaded and true image installed and then temporarily add the target block device to it in order to write the image, but I recently switched to using a WinPE boot ISO with GPLPV and TrueImage/Backup And Recovery installed, the Acronis products come with a WinPE/BartPE Image builder utility so its quite easy to do.

Also watch out for boot.ini, many servers come with a hidden partition on the boot volume for running diagnostics, in which case windows may be booting from the second partition, so you may need to adjust the boot.ini or you could remove it entirely and windows will then attempt to boot from \windows automatically.

Q (G5.21): Could we add a file image, block device or lvm to domU while it is running without restarting?

A (G5.21): Use "xm block-attach", and edit domU config file afterwards (if you still you file-based domU config) to make it permanent. Should work great with PV domUs.

There are cases when you have to reboot the domU anyway though, since the driver installation of a new disk on domU OS could require reboot. One example is when using Windows + GPLPV.

Devices

Q (G6.0): For my xen domUs I'm using a mixture of locah physical partitions (with LVM) and iSCSI disks. For local partitions, I don't have any problem, because LVM volumes are always the same. But for iSCSI disks, devices are assigned in the order they are connected, so I can't be sured that device that now is /dev/sdb (for example) will always be /dev/sdb. So, is there any way to identify the physical device in the domU configuration not as phy:/dev/sdb, but something like phy:label=fslabel? Or is there any other solution to this problem?

A (G6.0): I go with phy:/dev/disk/by-path/ip-*-iscsi-iqn.* If you assign iscsi luns directly as domU's fs without additional partitioning, you could probably also use /dev/disk/by-label/* or /dev/disk/by-uuid/*

Installation Questions

See Xen_FAQ_Installation

Networking Questions

See Xen_FAQ_Networking

Security Questions

See Xen_FAQ_Security

Design/Misc Questions

How does Xen Work

Q (D3.0): How does Xen process System Calls on para-virtualized guest?

A (D3.0): When ever a system call is invoked via interrupt or sys center control gets transferred to the kernel (ring 0), which is then handled via system call handler. System call never goes to libc but Libc is a library that provides POSIX interface to the user space applications and in a way wrapper for invocation of a system call. System call interrupt based [i386]: During booting process, linux kernel of a domU register's its IDT with Xen Hypervisor via HYPERVISOR_set_trap_table(trap_table); [arch/i386/kernel/traps-xen.c]. Xen maintains two IDT's, one global IDT (its own) and other per domain IDT. Xen uses global IDT to register the entire trap handler except for system call handler (int 0x80). When a VM gets scheduled, its system call handler (from per domain IDT table) is registered with the processor. Hence when a domain/VM executes a system call, its own handler is executed. Implementation differs for x86_64: Xen registers its own system call handler with the processor and from that handler routes the request to VM/Domain specific handler.

Q (D3.1): How does Xen process System Calls on fully virtualized guest (HVM)?

A (D3.1): For HVM domU there is no change in the behavior of the system call. HVM is only supported for Intel-VT and AMD-SVM processors. These processors are virtualization aware. Virtualization aware processors provide a new ring (Root-Ring 0) with higher privilege for VMM and Guest OS continues to runs with the same privilege (as without Xen) in Non-Root Ring 0. Guest OS can issue the system calls the way it used to without Xen.

Q (D3.2): Can I run various DomU operating systems on a different Dom0 operating system?

A (D3.2): Yes.

Q (D3.3): Just curious to know, if there is any way that given a terminal to a box, we can determine is it a physical machine or a virtual machine ?

A (D3.3): You should be able to get some useful information from the DMI, e.g: % for i in system-manufacturer system-product-name system-version\ system-serial-number; do echo -n "$i: "; sudo dmidecode -s $i; done system-manufacturer: Xen system-product-name: HVM domU system-version: 3.3.1 system-serial-number: 89e5915f-dead-beef-cefd-46904ea94c4a OR Probably checking kernel process, check your process table for: [xenwatch] [xenbus] Another clue is checking the kernle suffix, for example: and the proc files: /proc/xen/capabilities

Q (D3.4): What is STUBDOM ?

A (D3.4): Stubdoms are lightweight 'service' or 'driver' domains. The initial purpose was to offload qemu (for hvm guests) out of dom0. So with stubdoms you can run hvm guest qemu in a separate stubdom, which boosts performance and makes it more secure. stubdoms can also run for example pv-grub for pv guests, making it more secure compared to pygrub, which always runs in dom0. http://wiki.xensource.com/xenwiki/StubDom Presentation about stubdoms at Xen Summit: http://www.xen.org/files/xensummitboston08/SamThibault_XenSummit.pdf http://blog.xen.org/index.php/2008/08/28/xen-33-feature-stub-domains/ http://blog.xen.org/index.php/2008/08/28/xen-33-feature-hvm-device-model-domain/ http://lxr.xensource.com/lxr/source/stubdom/README

Q (D3.5): When is hardware virtualization used in Xen? Is it required?

A (D3.5): Xen uses hardware virtualization for HVM guests. Xen will not launch a HVM guest unless hardware virtualization is turned on.