Difference between revisions of "Fedora Host Installation"

From Xen
m
(Base System Installation)
 
(38 intermediate revisions by 4 users not shown)
Line 1: Line 1:
  +
__NOTOC__
This page covers the steps to get a working Xen host system (a.k.a. dom0) running [https://fedoraproject.org Fedora].
 
  +
{{sidebar
  +
| name = Content
   
  +
| outertitlestyle = text-align: left;
Fedora 16 is the first version of Fedora that comes with a kernel that supports a Xen dom0 out of the box since Fedora 8. This is a result of dom0 support being merged into the mainline kernel. As such, there are very few steps that have to be done to get a functional dom0. Some information about using a Fedora system as a virtualization host, coming directly from them, can be found [http://fedoraproject.org/wiki/Getting_started_with_virtualization here].
 
  +
| headingstyle = text-align: left;
  +
| contentstyle = text-align: left;
  +
  +
| content1 = __TOC__
  +
}}
  +
  +
This page covers the steps to get a working Xen Project host system (a.k.a. dom0) running [https://fedoraproject.org Fedora].
  +
  +
Fedora 16 is the first version of Fedora shipping a Linux kernel suitable for being used as Xen Project dom0 out of the box (the first since the time of Fedora 8). This comes directly from the fact that [http://blog.xenproject.org/index.php/2011/06/02/xen-celebrates-full-dom0-and-domu-support-in-linux-3-0/ dom0 support is now merged in mainline Linux]. This page explains the steps needed to turn a plain Fedora installation into fully functional Xen Project host.
  +
  +
Some general information about virtualization and Fedora are available [http://fedoraproject.org/wiki/Getting_started_with_virtualization here].
   
 
=Installing Fedora=
 
=Installing Fedora=
   
 
==Base System Installation==
 
==Base System Installation==
For the most part, you can follow the Installation steps outlined in the official Fedora installation guides (here they are for [http://docs.fedoraproject.org/en-US/Fedora/16/html/Installation_Guide/index.html F16], [http://docs.fedoraproject.org/en-US/Fedora/16/html/Installation_Guide/index.html F17] and [http://docs.fedoraproject.org/en-US/Fedora/18/html/Installation_Guide/index.html F18]). If you're lazy, you probably may just look at the installation quick start guides ([http://docs.fedoraproject.org/en-US/Fedora/16/html/Installation_Quick_Start_Guide/index.html F16], [http://docs.fedoraproject.org/en-US/Fedora/17/html/Installation_Quick_Start_Guide/index.html F17]).
+
For the most part, you can follow the Installation steps outlined in the official Fedora installation guides: [http://docs.fedoraproject.org/en-US/Fedora/18/html/Installation_Guide/index.html F18], [http://docs.fedoraproject.org/en-US/Fedora/19/html/Installation_Guide/ F19], and [http://docs.fedoraproject.org/en-US/Fedora/20/html/Installation_Guide/index.html F20]
   
  +
For the lazies, here they are the installation quick start guides: [http://docs.fedoraproject.org/en-US/Fedora/18/html/Installation_Quick_Start_Guide/index.html F18], [http://docs.fedoraproject.org/en-US/Fedora/19/html/Installation_Quick_Start_Guide/index.html F19], [http://docs.fedoraproject.org/en-US/Fedora/20/html/Installation_Quick_Start_Guide/index.html F20]
For obtaining install media images, look [http://fedoraproject.org/wiki/Distribution here].
 
   
  +
For obtaining install media images, look [https://fedoraproject.org/en/get-fedora here] or [http://fedoraproject.org/wiki/Distribution here].
The only portion you have to watch out is the disk partitioning section. If you're going to use file-backed domUs, you can safely ignore this step. However, if you're going to use LVM (or other block devices, like hard drive partitions)-backed domUs, you're probably going to partition your drive into /boot, and at least one or two volume groups, meaning that you'll be making a custom partitioning scheme. Have a look here for more details about this [[Host OS Install Considerations]].
 
  +
  +
The only portion one should watch out is the disk partitioning section. If wanting to to use file-backed domUs, nothing special even there. However, if LVM volumes (or other block devices, like hard drive partitions) is to be used, it is wise to partition the hard drive at least into a /boot physical partition, and one or two volume groups. Have a look [[Host OS Install Considerations|here]] for more details about this.
   
 
==Updating the System==
 
==Updating the System==
You probably want to be sure the all the software is as up-to-date as possible. To do that, run <pre> # yum update</pre> as root (or use any graphical front-end, e.g., find and click on 'Software Update').
+
You probably want to be sure the all the software is as up-to-date as possible. To do that, run <pre> # yum update</pre> (or use any graphical front-end, e.g., find and click on 'Software Update').
   
 
==Using the virt-preview Repository==
 
==Using the virt-preview Repository==
People who would like to test the very latest virtualization related Fedora packages (coming from Fedora rawhide) can enable the [http://fedoraproject.org/wiki/Virtualization_Preview_Repository Virtualization Preview Repository] (please, notice that Fedora discourages doing this in 'production' deployments).
+
People who would like to test the very latest virtualization related packages (coming from Fedora rawhide) can enable the [http://fedoraproject.org/wiki/Virtualization_Preview_Repository Virtualization Preview Repository] (please, notice that Fedora discourages doing this in 'production' deployments).
   
 
In order to make it happen, just do as follows:
 
In order to make it happen, just do as follows:
  +
<pre>
 
 
# cd /etc/yum.repos.d/
 
# cd /etc/yum.repos.d/
 
# wget http://fedorapeople.org/groups/virt/virt-preview/fedora-virt-preview.repo
 
# wget http://fedorapeople.org/groups/virt/virt-preview/fedora-virt-preview.repo
 
# yum update
 
# yum update
  +
</pre>
   
  +
==Disabling SELinux==
   
  +
Please report any SELinux issues in the Red Hat bugzilla system and on xen-devel mailing list.
=Installing and Running Xen=
 
  +
Fedora Core 21 and later should have most (all) issues resolved.
==Installing the Packages==
 
Getting the Xen hypervisor is a simple matter - just run
 
<pre> # yum install xen</pre> as root. Yum will take care of downloading the dependencies and installing them for you automatically.
 
   
  +
An work-around (if you have issues) is to:
As part of the download, Yum will create a new grub2 boot menu. As such, don't panic if Yum seems to freeze at "Installing xen-hypervisor" for a while.
 
  +
  +
<pre>
  +
# grep <vairous_stuff_here> /var/log/audit/audit.log | audit2allow -M mypol
  +
# semodule -i mypol.pp
  +
</pre>
  +
'''dozens''' of times!
  +
  +
=Installing and Running the Xen Project Hypervisor=
  +
==Installing the Packages==
  +
To install the Xen Project software, just run: <pre> # yum install xen</pre> as root. [http://fedoraproject.org/wiki/Yum yum] will, as usual, take care of downloading and installing anything that is needed. As a part of that, it also create a new [https://fedoraproject.org/wiki/GRUB_2 GRUB2] boot menu entry. Once the installation is complete, just reboot and select "Xen 4.x.x" from the GRUB2 menu.
   
  +
If on Fedora 18 and 19, you will likely see something like "Xen 4.2.x". If on Fedora 16 or 17, it will have "Xen 4.1.x" in it. It is also possible that you only see something like "Fedora, with Xen hypervisor", and that is also ok.
Once the installation is complete, you should reboot, and select Xen 4.x.x from the GRUB menu.
 
   
Networking deserves some special attention. In fact, it is something that gave a lot of trouble in the past (Xen 3/4.0). However, starting from Xen 4.1, the default behavior is to let the dom0's network scripts (i.e., the ones that came with the distribution, ''not'' the ones that come with Xen) handle setting up a network bridge, and let the toolstack just attach the domU's network interface to the it.
+
Networking deserves some special attention. In fact, it is something that gave a lot of trouble in the past (Xen 3/4.0). That is why, from Xen Project 4.1 on, the default behavior is to let the dom0's network scripts (i.e., the ones that came with the distribution, ''not'' the ones that come with Xen Project) handle setting up a network bridge, and let the toolstack just attach the domU's network interface to the it. More on this matter down in the page.
   
==Booting Xen by Default==
+
==Booting the Xen Project hypervisor by Default==
Although a Xen GRUB2 menu entry will be automatically created, it will likely not be the default boot option. If this is what you want, ensure to you have a line like this in <code>/etc/default/grub</code>:
+
Although a Xen Project GRUB2 menu entry will be automatically created, it will likely not be the default one. To make it so, ensure to you have a line like this in <code>/etc/default/grub</code>:
 
<pre>GRUB_DEFAULT="saved"</pre>
 
<pre>GRUB_DEFAULT="saved"</pre>
 
and then do:
 
and then do:
Line 44: Line 69:
 
# grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2
 
# grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2
 
# grub2-set-default <menu entry title you want>
 
# grub2-set-default <menu entry title you want>
  +
</pre>
  +
Typically, it would be something like this:
  +
<pre>
  +
# grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2
  +
Fedora, with Linux 3.11.3-201.fc19.x86_64
  +
Fedora, with Xen hypervisor
  +
# sudo grub2-set-default "Fedora, with Xen hypervisor"
 
</pre>
 
</pre>
   
 
==Disabling Network Manager==
 
==Disabling Network Manager==
In case of any networking issues, or if you want to '''manually''' manage the network configuration, disabling NetworkManager is necessary (as NetworkManager does not support bridges), and that requires manually editing a couple of config files. This is particularly true if you do not plan to install and use the typical Fedora virtualization tools, like [http://libvirt.org/ libvirt], etc. In fact, libvirt automatically creates a bridge and sets up the VMs created via its interface(s) to use it by default. If not installing libvirt, the bridge has to be created by hand.
+
In case of any networking issues, or if you want to '''manually''' manage the network configuration, disabling [https://fedoraproject.org/wiki/Tools/NetworkManager NetworkManager] is necessary (e.g., because NetworkManager does not support bridges), and that requires manually editing a couple of config files. This is particularly true if you do not plan to install and use the typical Fedora virtualization tools, like [http://libvirt.org/ libvirt], etc. In fact, libvirt automatically creates a bridge and sets up the VMs created via its interface(s) to use it by default. On the other hand, if not using libvirt, the bridge has to be created by hand.
 
For Red Hat-based distros, the networking configuration is stored in /etc/sysconfig/network-scripts/. By default, NetworkManager is included in the default Fedora install. To disable it, do the following (as root):
 
   
  +
For Red Hat-based distros, the networking configuration is stored in <code>/etc/sysconfig/network-scripts/</code>.
<pre> # systemctl disable NetworkManager.service || systemctl restart network.service</pre>
 
   
 
Also, make sure that the network service is kicked off automatically on subsequent restarts by running:
 
Also, make sure that the network service is kicked off automatically on subsequent restarts by running:
 
<pre> # chkconfig network on</pre>
 
<pre> # chkconfig network on</pre>
   
Next, create the configuration file for the bridge. The most common bridge name is br0, so let's stick to that (although other Xen documentation suggests using xenbr0 to make it obvious that the bridge is for Xen):
+
Next, create the configuration file for the bridge. The most common bridge name is br0, so let's stick to that (although other Xen Project documentation suggests using xenbr0 to make it obvious that the bridge is for the hypervisor):
 
 
<pre> # touch /etc/sysconfig/network-scripts/ifcfg-br0</pre>
 
<pre> # touch /etc/sysconfig/network-scripts/ifcfg-br0</pre>
   
Line 76: Line 105:
 
<pre> # systemctl restart network.service</pre>
 
<pre> # systemctl restart network.service</pre>
   
and your network should be looking like below:
+
and the bridge should look like this:
 
<pre>
 
<pre>
 
# ifconfig br0
 
# ifconfig br0
Line 88: Line 117:
 
RX bytes:329223 (321.5 KiB) TX bytes:76663 (74.8 KiB)</pre>
 
RX bytes:329223 (321.5 KiB) TX bytes:76663 (74.8 KiB)</pre>
   
== Rebooting and Installing DomUs ==
 
   
  +
Note that in earlier versions of Fedora (prior to 21), libvirt with xen would always assume
Now that Xen is installed, boot it and check it's working, e.g., by doing by running '''xl info''' from the command line. Your screen should then look like the below:
 
  +
that the default bridge was 'xenbr0'. And even if you did change the option (using the libvirt
  +
overrides) it would ignore it. Fedora 21 libvirt has that fixed.
   
  +
== Rebooting and "Running" the Xen Project hypervisor ==
  +
Now that the software is installed, (re)boot in the proper option, as explained above, and check it's working, e.g., by running <code>xl info</code> from the command line, which should then tell something like this:
 
<pre>
 
<pre>
# xl info
+
# xl info
host : caesium
+
host : localhost.localdomain
release : 3.1.1-2.fc16.i686.PAE
+
release : 3.7.6-201.fc18.x86_64
version : #1 SMP Mon Nov 14 15:57:20 UTC 2011
+
version : #1 SMP Mon Feb 4 15:54:08 UTC 2013
machine : i686
+
machine : x86_64
nr_cpus : 4
+
nr_cpus : 16
nr_nodes : 1
+
max_cpu_id : 63
  +
nr_nodes : 2
...
 
  +
cores_per_socket : 4
  +
threads_per_core : 2
  +
cpu_mhz : 2394
  +
hw_caps : bfebfbff:2c100800:00000000:00003f40:029ee3ff:00000000:00000001:00000000
  +
virt_caps : hvm hvm_directio
  +
total_memory : 12285
  +
free_memory : 128
  +
sharing_freed_memory : 0
  +
sharing_used_memory : 0
  +
free_cpus : 0
  +
xen_major : 4
  +
xen_minor : 2
  +
xen_extra : .1
  +
...
 
</pre>
 
</pre>
   
If Xen's not working, you'll get something like:
+
If the hypervisor is not working, you'll get something like:
  +
<pre>
 
<pre>libxl: error: libxl.c:56:libxl_ctx_init Is xenstore daemon running?
+
libxl: error: libxl.c:56:libxl_ctx_init Is xenstore daemon running?
 
failed to stat /var/run/xenstored.pid: No such file or directory
 
failed to stat /var/run/xenstored.pid: No such file or directory
cannot init xl context</pre>
+
cannot init xl context
  +
</pre>
   
  +
If you get something like the error message above, double check that all the needed servic are up and running and, if not, make sure they start automatically at boot. That is the case of <code>xenconsoled.service</code> and <code>xenstored.service</code> are enabled and running. You can check their status with <code>#systemctl status xenstored</code>. If it says something like "<code>inactive (dead)</code>", do the following:
By this point, your system should be set up to install domUs. You can install domUs with a variety of ways, e.g., by using xl. Notice that xl is the default toolstack since Xen 4.2, and therefore, <code>xend</code> is not started by default. If wanting to use <code>xm</code>/<code>xend</code>, you have to start the daemon manually, with something like: <pre> # xend</pre> but that will make it impossible to use <code>xl</code>.
 
  +
<pre>
  +
# systemctl enable xenstored
  +
# systemctl start xenstored
  +
</pre>
  +
(and the same for <code>xenconsoled</code>)
  +
  +
== Choosing a Toolstack ==
  +
By this point, everything is set, and ready to start installing VMs (also called domUs).
  +
  +
This can be done in a variety of ways, e.g., by using [[XL|xl]]. <code>xl</code> is the default toolstack since Xen Project 4.2 so, especially in F18, '''it is highly recommended to use it''', especially as the old toolstack, [[XEND|xend]] '''will be discontinued starting from Xen Project 4.3''' (for a comparison between the two, see [[XL vs Xend Feature Comparison|here]]). The two toolstacks do not play well together, and <code>xend</code> needs not to be running if wanting to issue <code>xl</code> commands. To check for that, something like:<pre> # ps aux | grep xend</pre> should do. If <code>xend</code> turns out to be running, it might be because it is being started automatically, as a service, during system boot. Check for that by doing:<pre> # systemctl is-enabled xend.service</pre> (and quite obviously the answer should be <code>disabled</code>), and check whether it currently running as a service with: <pre> # systemctl status xend.service</pre>
  +
To stop and disable it, and be then able to use <code>xl</code>, do:
  +
<pre>
  +
# systemctl stop xend.service
  +
# systemctl disable xend.service
  +
</pre>
  +
  +
Of course, if for any reason one wants to use <code>xend</code>, just do the opposite, i.e., enable and start it by:
  +
<pre>
  +
# systemctl enable xend.service
  +
# systemctl start xend.service
  +
</pre>
  +
  +
=Using xen-tools=
  +
[http://xen-tools.org/software/xen-tools/ xen-tools] is [http://blog.xenproject.org/index.php/2012/08/31/xen-tools-a-straightforward-vm-provisioninginstallation-tool/ "a straightforward Xen VM provisioning tool with an unusual but attractive approach"].
  +
  +
Although there is no RPM package for it, it is really easy to install and get it to work. Just do as follows (tested on Fedora 18):
  +
<pre> # yum install debootstrap perl-Text-Templateh perl-Config-IniFiles perl-File-Slurp perl-File-Which perl-Data-Dumper</pre>
  +
After that, download and install the upstream sources:
  +
<pre>
  +
$ wget http://xen-tools.org/software/xen-tools/xen-tools-4.3.1.tar.gz
  +
[...]
  +
$ tar zxvf xen-tools-4.3.1.tar.gz
  +
$ cd xen-tools-4.3.1
  +
# make install
  +
</pre>
   
  +
And it's done! A little bit more details on [[xen tools|Xen Project tools]] Wiki page and/or [http://blog.xenproject.org/index.php/2013/01/24/using-xen-tools-on-fedora/ this] blog post.
   
=Using libvirt and Typical Fedora Virt-Tools=
+
=Using libvirt and the Typical Fedora Virt-Tools=
 
==Installing the Packages==
 
==Installing the Packages==
 
If wanting to use [http://libvirt.org/ libvirt] and/or [http://virt-manager.org/ VirtManager], just do the following (tested on Fedora 18):
 
If wanting to use [http://libvirt.org/ libvirt] and/or [http://virt-manager.org/ VirtManager], just do the following (tested on Fedora 18):
 
<pre> # yum install libvirt-daemon-driver-xen python-virtinst libvirt-daemon-config-network libvirt-daemon-driver-network virt-manager virt-viewer</pre>
 
<pre> # yum install libvirt-daemon-driver-xen python-virtinst libvirt-daemon-config-network libvirt-daemon-driver-network virt-manager virt-viewer</pre>
   
'''Most likely''', everything will just work, and you are all set for starting creating VMs, e.g., with ''virt-install'' as described [[DomU Install with Virt-Install | here]] or with [http://virt-manager.org/ VirtManager], as described [[DomU Install with Virt-Manager | here]]. That happens because libvirt automatically creates and configures bridging.
+
'''Most likely''', everything will just work, and you are all set for starting creating VMs, e.g., with ''virt-install'' as described [[DomU Install with Virt-Install | here]] or with [http://virt-manager.org/ VirtManager], as described [[DomU Install with Virt-Manager | here]]. That happens because libvirt automatically creates and configures bridging and everything that is needed.
   
  +
A VNC client may reveal handy at some point, in which case, run: <pre> # yum install tigervnc</pre>
{{Warning| '''libvirt: libxl VS. xend'''
 
Starting from Xen 4.2, the default toolstack is libxl. libvirt has a libxl driver, but libvirt packages for F18 does not ship the proper version of it. On the other hand, the xend based driver is known to work quite well, so it might be best (for now) to just use that. That happens by making sure that ''libvirt-daemon-driver-libxl'' '''is not''' installed (again, tested on Fedora 18).
 
   
  +
===libvirt and libxl===
There is an on-going effort to fill the gap and render the libvirt libxl driver feature complete and fully functional, although it is yet unclear when that will happen.
 
  +
As already stated, starting from Xen Project 4.2, the default toolstack is libxl (i.e., the "library-counterpart" of <code>xl</code>). libvirt has a libxl driver, provided by the package ''libvirt-daemon-drive-libxl''. It should be installed as a dependency of ''libvirt-daemon-xen'', so it should already be there. The only other thing needed to use the libxl driver from virt-install and virt-manager is to make sure that '''<code>xend</code> is not running'''. In fact, if <code>xend</code> is running, the libvirt libxl driver will just disable itself, and the libvirt xend driver is what gets used.
   
  +
To stop and disable <code>xend</code>, follow the steps already provided above.
This page will be kept up-to-date with any progress being made with respect to this, so come back here periodically and check them out.}}
 
   
==Installing DomUs==
+
===libvirt and xend===
  +
If in Fedora 18, libvirt libxl driver is the default, if installed and if <code>xend</code> is not running. However, just by having xend running, it is still possible to use the libvirt xend driver. It is, again, enough to have <code>xend</code> up and running and to have the package '''libvirt-daemon-driver-xen'''.
With libvirt, installing VMs, especially in case of Red Hat based distros (like Fedora/CentOS), is extremely easy. You can refer to:
 
  +
  +
==Running the domUs==
  +
With libvirt, installing VMs (especially in case of RedHat based distros, like Fedora or CentOS), this extremely easy, and you can refer to:
 
* [[DomU_Install_with_Virt-Install|installing a domU with virt-install]] (console based) or,
 
* [[DomU_Install_with_Virt-Install|installing a domU with virt-install]] (console based) or,
* [[DomU_Install_with_Virt-Manager|installing a domU with virt-manager]] (GUI based).
+
* [[DomU_Install_with_Virt-Manager|installing a domU with virt-manager]] (GUI based),
  +
for more details.
   
 
=Known Issues=
 
=Known Issues=
   
 
=== GRUB2 Menu in Fedora 16 ===
 
=== GRUB2 Menu in Fedora 16 ===
In Fedora 16 there is a [https://bugzilla.redhat.com/show_bug.cgi?id=738085 known bug] that results in GRUB2 generating a number of boot entries for each Xen file that it finds in /boot. Unfortunately, it doesn't check if the file in question is a linked file, and results in generating a host of spurious entries. This is no longer the case from Fedora 17.
+
In Fedora 16 there is a [https://bugzilla.redhat.com/show_bug.cgi?id=738085 known bug] that results in GRUB2 generating a number of boot entries for each Xen Project file that it finds in /boot, even if they are just symlinks. This results in quite a bit of spurious entries. This is no longer the case starting from Fedora 17.
   
 
=== SELinux and LVM in Fedora 16 & 17 ===
 
=== SELinux and LVM in Fedora 16 & 17 ===
In Fedora 16 and 17, you may also need to [http://docs.fedoraproject.org/en-US/Fedora/13/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-Working_with_SELinux-Main_Configuration_File.html disable SELinux] if you're managing LVM storage manually, unless you want to relabel the created LVs. xl '''will''' complain that ''"the disk is not accessible"'' if you fail to do this and you're using LVs for storage. This does not seem to be an issue in Fedora 18 (if it is for you, report it to [http://www.redhat.com/mailman/listinfo/fedora-virt fedora-virt] and/or [http://www.redhat.com/mailman/listinfo/fedora-xen fedora-xen]).
+
In Fedora 16 and 17, you may need to [http://docs.fedoraproject.org/en-US/Fedora/13/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-Working_with_SELinux-Main_Configuration_File.html disable SELinux] if you're managing LVM storage manually (unless you want to relabel the volumes). In fact, <code>xl</code> will otherwise complain that ''"the disk is not accessible"''.
   
  +
This does not seem to be an issue in Fedora 18 (if it is for you, report it to [http://www.redhat.com/mailman/listinfo/fedora-virt fedora-virt] and/or [http://www.redhat.com/mailman/listinfo/fedora-xen fedora-xen]). Of course, SELinux may cause a number of other issues (for instance if using <code>xend</code>) even in F18, thus disabling it might well be the best option anyway.
=== Kernel Update in Fedora 16, 17 & 18 ===
 
  +
When a new kernel is installed, as a consequence of running: <pre> # yum update</pre> [http://git.fedorahosted.org/git/grubby.git grubby] is used to update the GRUB2 config file. As reported by this [https://bugzilla.redhat.com/show_bug.cgi?id=783851 bug], grubby does not handle Xen menu entries very well, and thus it is possible for Xen to no longer be able to boot. We're looking into possible fixes. For now, the solution is to '''manually run''', after each update that install a new kernel, the following command: <pre># grub2-mkconfig -o /boot/grub2/grub.cfg</pre>
 
  +
=== Kernel Update in Fedora 16, 17, 18 and 19 ===
  +
When a new kernel is installed, as a consequence of running: <pre> # yum update</pre> [http://git.fedorahosted.org/git/grubby.git grubby] is used to update the GRUB2 config file. As reported by this [https://bugzilla.redhat.com/show_bug.cgi?id=783851 bug], grubby does not handle Xen Project menu entries very well, and thus it is possible for Xen Project to no longer be able to boot. We're looking into possible fixes. For now, the solution is to '''manually run''', after each update that install a new kernel, the following command: <pre># grub2-mkconfig -o /boot/grub2/grub.cfg</pre>
   
 
That is why, when updating the xen-hypervisor package, it is <code>grub2-mkconfig</code> that is invoked, and hence the issue does not manifest itself in that case.
 
That is why, when updating the xen-hypervisor package, it is <code>grub2-mkconfig</code> that is invoked, and hence the issue does not manifest itself in that case.
   
=== xend vs. libxl in Fedora 16, 17 & 18 ===
+
=== libvirt, xend and libxl in Fedora 16, 17 & 18 ===
Starting from Xen 4.2, the default toolstack is libxl. libvirt has a libxl driver, but libvirt packages does not ship the proper version of it. On the other hand, the xend based driver is known to work quite well, so it is best (for now) to just use it. That happens by making sure that libvirt-daemon-driver-libxl is not installed (tested recently on Fedora 18).
+
libxl is the default toolstack for Xen Project 4.2, while it is nothing more than a tech preview for previous releases of Xen Project. For that reason, the libvirt libxl driver is disabled in Fedora 16 and 17, and the only option to use virt-install and virt-manager there, is to run and use <code>xend</code>.
  +
  +
There was an issue with default bridge always having to be xenbr0, see [https://bugzilla.redhat.com/show_bug.cgi?id=978443 virt-install without -w bridge=<name> using xen is unable to start] which
  +
is fixed in Fedora 21.
   
  +
=== libvirt, xl and libxl in Fedora 21 ===
There is an on-going effort to fill the gap and render the libvirt libxl driver feature complete and fully functional, although it is yet unclear when that will happen.
 
   
  +
libxl is the default toolstack for Xen Project 4.4. If using 'libvirt' tools make sure you have the proper driver installed: ''libvirt-daemon-driver-xen''
=Running Xen and Installing DomUs=
 
   
 
[[Category:Xen]]
 
[[Category:Xen]]
Line 157: Line 249:
 
[[Category:Host Install]]
 
[[Category:Host Install]]
 
[[Category:Fedora]]
 
[[Category:Fedora]]
  +
[[Category:Fundamentals]]
  +
[[Category:Libvirt]]

Latest revision as of 17:01, 11 November 2014

This page covers the steps to get a working Xen Project host system (a.k.a. dom0) running Fedora.

Fedora 16 is the first version of Fedora shipping a Linux kernel suitable for being used as Xen Project dom0 out of the box (the first since the time of Fedora 8). This comes directly from the fact that dom0 support is now merged in mainline Linux. This page explains the steps needed to turn a plain Fedora installation into fully functional Xen Project host.

Some general information about virtualization and Fedora are available here.

Installing Fedora

Base System Installation

For the most part, you can follow the Installation steps outlined in the official Fedora installation guides: F18, F19, and F20

For the lazies, here they are the installation quick start guides: F18, F19, F20

For obtaining install media images, look here or here.

The only portion one should watch out is the disk partitioning section. If wanting to to use file-backed domUs, nothing special even there. However, if LVM volumes (or other block devices, like hard drive partitions) is to be used, it is wise to partition the hard drive at least into a /boot physical partition, and one or two volume groups. Have a look here for more details about this.

Updating the System

You probably want to be sure the all the software is as up-to-date as possible. To do that, run

 # yum update

(or use any graphical front-end, e.g., find and click on 'Software Update').

Using the virt-preview Repository

People who would like to test the very latest virtualization related packages (coming from Fedora rawhide) can enable the Virtualization Preview Repository (please, notice that Fedora discourages doing this in 'production' deployments).

In order to make it happen, just do as follows:

 # cd /etc/yum.repos.d/
 # wget http://fedorapeople.org/groups/virt/virt-preview/fedora-virt-preview.repo
 # yum update

Disabling SELinux

Please report any SELinux issues in the Red Hat bugzilla system and on xen-devel mailing list. Fedora Core 21 and later should have most (all) issues resolved.

An work-around (if you have issues) is to:

 # grep <vairous_stuff_here> /var/log/audit/audit.log | audit2allow -M mypol
 # semodule -i mypol.pp

dozens of times!

Installing and Running the Xen Project Hypervisor

Installing the Packages

To install the Xen Project software, just run:

 # yum install xen

as root. yum will, as usual, take care of downloading and installing anything that is needed. As a part of that, it also create a new GRUB2 boot menu entry. Once the installation is complete, just reboot and select "Xen 4.x.x" from the GRUB2 menu.

If on Fedora 18 and 19, you will likely see something like "Xen 4.2.x". If on Fedora 16 or 17, it will have "Xen 4.1.x" in it. It is also possible that you only see something like "Fedora, with Xen hypervisor", and that is also ok.

Networking deserves some special attention. In fact, it is something that gave a lot of trouble in the past (Xen 3/4.0). That is why, from Xen Project 4.1 on, the default behavior is to let the dom0's network scripts (i.e., the ones that came with the distribution, not the ones that come with Xen Project) handle setting up a network bridge, and let the toolstack just attach the domU's network interface to the it. More on this matter down in the page.

Booting the Xen Project hypervisor by Default

Although a Xen Project GRUB2 menu entry will be automatically created, it will likely not be the default one. To make it so, ensure to you have a line like this in /etc/default/grub:

GRUB_DEFAULT="saved"

and then do:

 # grub2-mkconfig -o /boot/grub2/grub.cfg
 # grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2
 # grub2-set-default <menu entry title you want>

Typically, it would be something like this:

 # grep ^menuentry /boot/grub2/grub.cfg | cut -d "'" -f2
  Fedora, with Linux 3.11.3-201.fc19.x86_64
  Fedora, with Xen hypervisor
 # sudo grub2-set-default "Fedora, with Xen hypervisor"

Disabling Network Manager

In case of any networking issues, or if you want to manually manage the network configuration, disabling NetworkManager is necessary (e.g., because NetworkManager does not support bridges), and that requires manually editing a couple of config files. This is particularly true if you do not plan to install and use the typical Fedora virtualization tools, like libvirt, etc. In fact, libvirt automatically creates a bridge and sets up the VMs created via its interface(s) to use it by default. On the other hand, if not using libvirt, the bridge has to be created by hand.

For Red Hat-based distros, the networking configuration is stored in /etc/sysconfig/network-scripts/.

Also, make sure that the network service is kicked off automatically on subsequent restarts by running:

 # chkconfig network on

Next, create the configuration file for the bridge. The most common bridge name is br0, so let's stick to that (although other Xen Project documentation suggests using xenbr0 to make it obvious that the bridge is for the hypervisor):

 # touch /etc/sysconfig/network-scripts/ifcfg-br0

open the file in a text editor and put the following lines in it:

DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Next, you need to find the configuration file for your existing network adaptor. It's most likely named ifcfg-eth0 or ifcfg-em0 (the part after the dash can and does change depending on the actual hardware). Once you know what config file you're using, open it, find the line

NM_CONTROLLED=yes

and change it to

NM_CONTROLLED=no

. Also, do not forget to add the line

BRIDGE=br0

(or whatever you named your bridge) to that file.

Finally, run this:

 # systemctl restart network.service

and the bridge should look like this:

 # ifconfig br0
 br0       Link encap:Ethernet  HWaddr 00:07:E9:06:50:12
           inet addr:192.168.1.122  Bcast:192.168.1.255  Mask:255.255.255.0
           inet6 addr: fe80::207:e9ff:fe06:5012/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:1858 errors:0 dropped:0 overruns:0 frame:0
           TX packets:678 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:0
           RX bytes:329223 (321.5 KiB)  TX bytes:76663 (74.8 KiB)


Note that in earlier versions of Fedora (prior to 21), libvirt with xen would always assume that the default bridge was 'xenbr0'. And even if you did change the option (using the libvirt overrides) it would ignore it. Fedora 21 libvirt has that fixed.

Rebooting and "Running" the Xen Project hypervisor

Now that the software is installed, (re)boot in the proper option, as explained above, and check it's working, e.g., by running xl info from the command line, which should then tell something like this:

 # xl info
 host                   : localhost.localdomain
 release                : 3.7.6-201.fc18.x86_64
 version                : #1 SMP Mon Feb 4 15:54:08 UTC 2013
 machine                : x86_64
 nr_cpus                : 16
 max_cpu_id             : 63
 nr_nodes               : 2
 cores_per_socket       : 4
 threads_per_core       : 2
 cpu_mhz                : 2394
 hw_caps                : bfebfbff:2c100800:00000000:00003f40:029ee3ff:00000000:00000001:00000000
 virt_caps              : hvm hvm_directio
 total_memory           : 12285
 free_memory            : 128
 sharing_freed_memory   : 0
 sharing_used_memory    : 0
 free_cpus              : 0
 xen_major              : 4
 xen_minor              : 2
 xen_extra              : .1
 ...

If the hypervisor is not working, you'll get something like:

libxl: error: libxl.c:56:libxl_ctx_init Is xenstore daemon running?
failed to stat /var/run/xenstored.pid: No such file or directory
cannot init xl context

If you get something like the error message above, double check that all the needed servic are up and running and, if not, make sure they start automatically at boot. That is the case of xenconsoled.service and xenstored.service are enabled and running. You can check their status with #systemctl status xenstored. If it says something like "inactive (dead)", do the following:

# systemctl enable xenstored
# systemctl start xenstored

(and the same for xenconsoled)

Choosing a Toolstack

By this point, everything is set, and ready to start installing VMs (also called domUs).

This can be done in a variety of ways, e.g., by using xl. xl is the default toolstack since Xen Project 4.2 so, especially in F18, it is highly recommended to use it, especially as the old toolstack, xend will be discontinued starting from Xen Project 4.3 (for a comparison between the two, see here). The two toolstacks do not play well together, and xend needs not to be running if wanting to issue xl commands. To check for that, something like:

 # ps aux | grep xend

should do. If xend turns out to be running, it might be because it is being started automatically, as a service, during system boot. Check for that by doing:

 # systemctl is-enabled xend.service

(and quite obviously the answer should be disabled), and check whether it currently running as a service with:

 # systemctl status xend.service

To stop and disable it, and be then able to use xl, do:

 # systemctl stop xend.service
 # systemctl disable xend.service

Of course, if for any reason one wants to use xend, just do the opposite, i.e., enable and start it by:

 # systemctl enable xend.service
 # systemctl start xend.service

Using xen-tools

xen-tools is "a straightforward Xen VM provisioning tool with an unusual but attractive approach".

Although there is no RPM package for it, it is really easy to install and get it to work. Just do as follows (tested on Fedora 18):

 # yum install debootstrap perl-Text-Templateh perl-Config-IniFiles perl-File-Slurp perl-File-Which perl-Data-Dumper

After that, download and install the upstream sources:

 $ wget http://xen-tools.org/software/xen-tools/xen-tools-4.3.1.tar.gz
 [...]
 $ tar zxvf xen-tools-4.3.1.tar.gz
 $ cd xen-tools-4.3.1
 # make install

And it's done! A little bit more details on Xen Project tools Wiki page and/or this blog post.

Using libvirt and the Typical Fedora Virt-Tools

Installing the Packages

If wanting to use libvirt and/or VirtManager, just do the following (tested on Fedora 18):

 # yum install libvirt-daemon-driver-xen python-virtinst libvirt-daemon-config-network libvirt-daemon-driver-network virt-manager virt-viewer

Most likely, everything will just work, and you are all set for starting creating VMs, e.g., with virt-install as described here or with VirtManager, as described here. That happens because libvirt automatically creates and configures bridging and everything that is needed.

A VNC client may reveal handy at some point, in which case, run:

 # yum install tigervnc

libvirt and libxl

As already stated, starting from Xen Project 4.2, the default toolstack is libxl (i.e., the "library-counterpart" of xl). libvirt has a libxl driver, provided by the package libvirt-daemon-drive-libxl. It should be installed as a dependency of libvirt-daemon-xen, so it should already be there. The only other thing needed to use the libxl driver from virt-install and virt-manager is to make sure that xend is not running. In fact, if xend is running, the libvirt libxl driver will just disable itself, and the libvirt xend driver is what gets used.

To stop and disable xend, follow the steps already provided above.

libvirt and xend

If in Fedora 18, libvirt libxl driver is the default, if installed and if xend is not running. However, just by having xend running, it is still possible to use the libvirt xend driver. It is, again, enough to have xend up and running and to have the package libvirt-daemon-driver-xen.

Running the domUs

With libvirt, installing VMs (especially in case of RedHat based distros, like Fedora or CentOS), this extremely easy, and you can refer to:

for more details.

Known Issues

GRUB2 Menu in Fedora 16

In Fedora 16 there is a known bug that results in GRUB2 generating a number of boot entries for each Xen Project file that it finds in /boot, even if they are just symlinks. This results in quite a bit of spurious entries. This is no longer the case starting from Fedora 17.

SELinux and LVM in Fedora 16 & 17

In Fedora 16 and 17, you may need to disable SELinux if you're managing LVM storage manually (unless you want to relabel the volumes). In fact, xl will otherwise complain that "the disk is not accessible".

This does not seem to be an issue in Fedora 18 (if it is for you, report it to fedora-virt and/or fedora-xen). Of course, SELinux may cause a number of other issues (for instance if using xend) even in F18, thus disabling it might well be the best option anyway.

Kernel Update in Fedora 16, 17, 18 and 19

When a new kernel is installed, as a consequence of running:

 # yum update

grubby is used to update the GRUB2 config file. As reported by this bug, grubby does not handle Xen Project menu entries very well, and thus it is possible for Xen Project to no longer be able to boot. We're looking into possible fixes. For now, the solution is to manually run, after each update that install a new kernel, the following command:

# grub2-mkconfig -o /boot/grub2/grub.cfg

That is why, when updating the xen-hypervisor package, it is grub2-mkconfig that is invoked, and hence the issue does not manifest itself in that case.

libvirt, xend and libxl in Fedora 16, 17 & 18

libxl is the default toolstack for Xen Project 4.2, while it is nothing more than a tech preview for previous releases of Xen Project. For that reason, the libvirt libxl driver is disabled in Fedora 16 and 17, and the only option to use virt-install and virt-manager there, is to run and use xend.

There was an issue with default bridge always having to be xenbr0, see virt-install without -w bridge=<name> using xen is unable to start which is fixed in Fedora 21.

libvirt, xl and libxl in Fedora 21

libxl is the default toolstack for Xen Project 4.4. If using 'libvirt' tools make sure you have the proper driver installed: libvirt-daemon-driver-xen