Difference between revisions of "Securing Xen"

From Xen
(Migrated page)
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  +
This document describes how to secure a Xen system. It describes a number of advanced Xen features which can be used to increase the security of a system.
<!-- MoinMoin name: SecuringXen -->
 
  +
__TOC__
<!-- Comment: -->
 
  +
[[File:Xen-Panda-Security-500px.jpg|right|150px]]
<!-- WikiMedia name: SecuringXen -->
 
  +
= Introduction =
<!-- Page revision: 00000001 -->
 
  +
Xen has a number of advanced security features, designed to allow users of Xen to build a more secure set-up. Most of them
<!-- Original date: Mon Oct 31 20:25:27 2005 (1130790327000000) -->
 
  +
* are switched off by default
  +
* some need to be enabled by re-compiling Xen
  +
A large number of these features are quite simple to use. Note that there is a very good introduction in [http://www.slideshare.net/xen_com_mgr/a-brief-tutorial-on-xens-advanced-security-features this slidedeck]. The following table gives an overview also:
   
  +
{| class="zebra fullwidth sortable"
{{TODO|This document is old, but more security content is needed}}
 
  +
|-
  +
! Threat(s) / Attack Surface / Use
  +
! Security Feature / Practice
  +
! Disabled by Default
  +
! Easy to set up
  +
! Requires re-compile
  +
|-
  +
| Network || [[Driver Domain]] || Yes || Yes || No
  +
|-
  +
| Guest Boot || [http://www.slideshare.net/xen_com_mgr/a-brief-tutorial-on-xens-advanced-security-features/16 Fixed Kernels] || N/A || Yes || N/A
  +
|-
  +
| Guest Boot || [[PvGrub]] || Yes || Yes || No
  +
|-
  +
| Device Model / QEMU || [[Device Model Stub Domains]] || Yes || Yes || No
  +
|-
  +
| Wide Range || [[Xen Security Modules : XSM-FLASK]] || Yes || Need to be familiar with SELinux || Yes
  +
|-
  +
| Crypto Key Storage || [http://xenbits.xen.org/docs/unstable/misc/vtpm.txt Virtual TPM] || Yes || Need to be familiar with TPM || May require extra packages
  +
|}
   
  +
= Advanced Xen Security Features =
__NOTOC__
 
= Securing Xen =
 
   
  +
== Basic setup ==
This chapter describes how to secure a Xen system. It describes a number
 
of scenarios and provides a corresponding set of best practices. It begins
 
with a section devoted to understanding the security implications of a Xen
 
system.
 
   
  +
The basic setup we'll be considering is as follows:
== Xen Security Considerations ==
 
  +
* A system with an IOMMU with interrupt re-routing.
  +
** For AMD systems, this should be any system with an IOMMU.
  +
** For Intel systems, this should be any system with VT-d version 2 or later.
  +
* Two network cards, attached to the following networks:
  +
** '''control network''': isolated from the internet and from guests. dom0 listens on this network
  +
** '''guest network''': Access to guests and the internet. Domain 0 not accessible from this network.
   
  +
All the general advice for servers, regarding starting as few services as possible and firewalling off ports applies.
When deploying a Xen system, one must be sure to secure the management
 
domain (Domain-0) as much as possible. If the management domain is
 
comprimised, all other domains are also vunerable. The following are
 
a set of best practices for Domain-0:
 
   
  +
For the purposes of this page, we assume that an attacker has (or can get) complete control over the kernel of one of the guest VMs. We also assume that she can inject arbitrary packets into the guest network (either from a VM or from the internet). Our goal is to keep her from being able to read or modify the memory of other guests.
# '''Run the smallest number of necessary services''' the less things that are present in management partition the better. Remember, a service running as root in the management domain has full access to all other domains on the system.
 
# '''Use a firewall to restrict the traffic to the management domain''' a firewall with default-reject rules will help prevent attacks on the management domain.
 
# '''Do not allow users to access Domain-0''' the Linux kernel has been known to have local-user root exploits. If you allow normal users to access Domain-0 (even as unprivileged users) you run the risk of a kernel exploiting making all of your domains vunerable.
 
   
  +
== Network driver: Driver domains ==
== Security Scenarios ==
 
   
  +
The first attack surface to consider is the networking infrastructure. This includes the hardware driver for the network interface card, as well as the bridging software (Linux bridge-utils or open vswitch), and the network backend.
=== The Isolated Management Network ===
 
   
  +
The best way to secure this is to use a driver domain for the guest network. Information on how to set up a driver domain can be found here: [[Driver_Domain]]
In this scenario, each node has two network cards in the cluster. One network card is connected to the outside world and one network card is a physically isolated management network specifically for Xen instances to use.
 
   
  +
== Secure grub for pv domains: pvgrub ==
As long as all of the management partitions are trusted equally, this is the most secure scenario. No additional configuration is needed other than forcing Xend to bind to the management interface for relocation.
 
   
  +
The second attack surface to consider is in respect to PV booting. The default for PV domains is for dom0 to pass the kernel to the domain builder. This is secure, but requires the administrator to keep up to date with kernel updates. It is also less convenient for the user, who can now no longer choose their own kernel parameters.
'''FIXME:''' what is the option to allow for this?
 
   
  +
The first alternative is to use pygrub. Pygrub is a python program that runs in domain 0. It reads the guest filesystem, finds the grub config files, interprets them to present a menu, then reads the selected file off the guest disk and passes it to the domain zero.
=== A Subnet Behind a Firewall ===
 
   
  +
The problem is that pygrub and the domain builder run in domain 0, the builder with full privileges to anything on the system. This makes them a juicy target for an attacker; any bug in python, the grub interpreter, the filesystem library, or the domain builder could be exploited to get full control of the system.
In this scenario, each node has only one network card but the entire cluster sits behind a firewall. This firewall should do the following (in the least):
 
   
  +
The solution is to use pvgrub, a version of grub ported to run on minios. When using pvgrub, the domain builder will build the domain with the full amount of memory, but will start an image containing just minios and pvgrub. pvgrub runs inside the new guest and will display the menu, and load and kexec the appropriate kernel.
# Prevent IP spoofing from outside of the Subnet
 
# Prevent access to the relocation port of any of the nodes in the cluster except from within the cluster[1].
 
   
  +
Because pvgrub runs inside the guest context, it has no more privileges than the guest already has; there is therefore no benefit to attacking it.
[1] The following iptables rules can be used on each node to prevent migrations to that node from outside the subnet assuming the main firewall does not do this for you:
 
   
  +
pvgrub can be run by simply setting the kernel in the guest config to point to the appropriate pvgrub image. Instructions for doing can be found here: [[PvGrub]]
   
  +
== Securing HVM domains: device model stub domains ==
<pre><nowiki>
 
# this command disables all access to the Xen relocation port
 
iptables -A INPUT -p tcp --destination-port 8002 -j REJECT
 
   
  +
The next attack target to consider is qemu. HVM domains use qemu to provide emulated hardware (also known as the "device model"), typically including a motherboard, pci bus, mouse, cdrom, and sometimes disk or network controllers as well.
# this command enables Xen relocations only from the specific subnet
 
iptables -I INPUT -p tcp --source 192.168.1.1/8 --destination-port 8002 -j ACCEPT
 
</nowiki></pre>
 
   
  +
In the default configuration, qemu runs inside of dom0, and has permission to read or write from any VM on the system, again providing an attractive target. Hardware devices can have complicated interfaces, and the code is not always written with security in mind.
   
  +
The solution is to run "device model stub domains". A stub domain is a domain with a minimal OS designed to run a single application; in this case, qemu.
=== Nodes on an Untrusted Subnet ===
 
   
  +
Using device models is as simple as changing a setting in the config file. You can read more about setting them up here: [[Device_Model_Stub_Domains]].
Migration on an untrusted subnet is not safe in current versions of Xen. It may be possible to perform migrations through a secure tunnel (through a VPN or SSH) however this is not described here.
 
   
  +
== Experimental: Using FLASK's example policy ==
The only safe option in this scenario is to disable migration completely. The easiest way to do this is with iptables:
 
   
  +
We've discussed securing various codebases outside of the hypervisor. But what about Xen itself? Both PV and HVM guests can attack Xen itself through the hypercall interface. While the Xen developers try to be very careful with the security of the hypercall interface, and now regularly do "fuzz testing" on it to discover potential vulnerabilities, it is inevitable that they will make some mistakes.
   
  +
One of the ways to mitigate mistakes in the hypercall layer is to use the Xen Security Module (XSM) with FLASK. FLASK is a plugin for XSM written by the NSA, and is Xen's analog of the SELinux security framework. It allows you to load policies to restrict what hypercalls a VM can try to make. Although this doesn't guarantee that no vulnerabilities will be exposed, by restricting what hypercalls are valid, it reduces the probability of an exploitable vulnerability significantly.
<pre><nowiki>
 
  +
# this command disables all access to the Xen relocation port
 
  +
FLASK is a mature piece of software, but writing policies is a very tricky and difficult task -- not for the faint of heart. Xen does come with a useful set of "example" policies, which cover most roles that a typical installation would need to use; including driver domains, normal domains, stub domains, and so on. However, these example policies are not heavily tested; they should be used at your own risk.
iptables -A INPUT -p tcp --destination-port 8002 -j REJECT
 
  +
</nowiki></pre>
 
  +
The basic steps to use FLASK's example policies are as follows:
  +
* Build Xen with XSM enabled
  +
* Build the example policy
  +
* Add the appropriate "labels" to the VM config files, using <code>seclabel=[$foo]</code>, based on their role.
  +
  +
For more information, please see [[Xen_Security_Modules_:_XSM-FLASK]]
  +
  +
== Also See ==
  +
* Presentation: [http://www.slideshare.net/xen_com_mgr/a-brief-tutorial-on-xens-advanced-security-features Securing your cloud with Xen's advanced security features]
   
 
[[Category:Xen]]
 
[[Category:Xen]]
  +
[[Category:HowTo]]
 
[[Category:Security]]
 
[[Category:Security]]

Revision as of 09:22, 18 October 2013

This document describes how to secure a Xen system. It describes a number of advanced Xen features which can be used to increase the security of a system.

Xen-Panda-Security-500px.jpg

Introduction

Xen has a number of advanced security features, designed to allow users of Xen to build a more secure set-up. Most of them

  • are switched off by default
  • some need to be enabled by re-compiling Xen

A large number of these features are quite simple to use. Note that there is a very good introduction in this slidedeck. The following table gives an overview also:

Threat(s) / Attack Surface / Use Security Feature / Practice Disabled by Default Easy to set up Requires re-compile
Network Driver Domain Yes Yes No
Guest Boot Fixed Kernels N/A Yes N/A
Guest Boot PvGrub Yes Yes No
Device Model / QEMU Device Model Stub Domains Yes Yes No
Wide Range Xen Security Modules : XSM-FLASK Yes Need to be familiar with SELinux Yes
Crypto Key Storage Virtual TPM Yes Need to be familiar with TPM May require extra packages

Advanced Xen Security Features

Basic setup

The basic setup we'll be considering is as follows:

  • A system with an IOMMU with interrupt re-routing.
    • For AMD systems, this should be any system with an IOMMU.
    • For Intel systems, this should be any system with VT-d version 2 or later.
  • Two network cards, attached to the following networks:
    • control network: isolated from the internet and from guests. dom0 listens on this network
    • guest network: Access to guests and the internet. Domain 0 not accessible from this network.

All the general advice for servers, regarding starting as few services as possible and firewalling off ports applies.

For the purposes of this page, we assume that an attacker has (or can get) complete control over the kernel of one of the guest VMs. We also assume that she can inject arbitrary packets into the guest network (either from a VM or from the internet). Our goal is to keep her from being able to read or modify the memory of other guests.

Network driver: Driver domains

The first attack surface to consider is the networking infrastructure. This includes the hardware driver for the network interface card, as well as the bridging software (Linux bridge-utils or open vswitch), and the network backend.

The best way to secure this is to use a driver domain for the guest network. Information on how to set up a driver domain can be found here: Driver_Domain

Secure grub for pv domains: pvgrub

The second attack surface to consider is in respect to PV booting. The default for PV domains is for dom0 to pass the kernel to the domain builder. This is secure, but requires the administrator to keep up to date with kernel updates. It is also less convenient for the user, who can now no longer choose their own kernel parameters.

The first alternative is to use pygrub. Pygrub is a python program that runs in domain 0. It reads the guest filesystem, finds the grub config files, interprets them to present a menu, then reads the selected file off the guest disk and passes it to the domain zero.

The problem is that pygrub and the domain builder run in domain 0, the builder with full privileges to anything on the system. This makes them a juicy target for an attacker; any bug in python, the grub interpreter, the filesystem library, or the domain builder could be exploited to get full control of the system.

The solution is to use pvgrub, a version of grub ported to run on minios. When using pvgrub, the domain builder will build the domain with the full amount of memory, but will start an image containing just minios and pvgrub. pvgrub runs inside the new guest and will display the menu, and load and kexec the appropriate kernel.

Because pvgrub runs inside the guest context, it has no more privileges than the guest already has; there is therefore no benefit to attacking it.

pvgrub can be run by simply setting the kernel in the guest config to point to the appropriate pvgrub image. Instructions for doing can be found here: PvGrub

Securing HVM domains: device model stub domains

The next attack target to consider is qemu. HVM domains use qemu to provide emulated hardware (also known as the "device model"), typically including a motherboard, pci bus, mouse, cdrom, and sometimes disk or network controllers as well.

In the default configuration, qemu runs inside of dom0, and has permission to read or write from any VM on the system, again providing an attractive target. Hardware devices can have complicated interfaces, and the code is not always written with security in mind.

The solution is to run "device model stub domains". A stub domain is a domain with a minimal OS designed to run a single application; in this case, qemu.

Using device models is as simple as changing a setting in the config file. You can read more about setting them up here: Device_Model_Stub_Domains.

Experimental: Using FLASK's example policy

We've discussed securing various codebases outside of the hypervisor. But what about Xen itself? Both PV and HVM guests can attack Xen itself through the hypercall interface. While the Xen developers try to be very careful with the security of the hypercall interface, and now regularly do "fuzz testing" on it to discover potential vulnerabilities, it is inevitable that they will make some mistakes.

One of the ways to mitigate mistakes in the hypercall layer is to use the Xen Security Module (XSM) with FLASK. FLASK is a plugin for XSM written by the NSA, and is Xen's analog of the SELinux security framework. It allows you to load policies to restrict what hypercalls a VM can try to make. Although this doesn't guarantee that no vulnerabilities will be exposed, by restricting what hypercalls are valid, it reduces the probability of an exploitable vulnerability significantly.

FLASK is a mature piece of software, but writing policies is a very tricky and difficult task -- not for the faint of heart. Xen does come with a useful set of "example" policies, which cover most roles that a typical installation would need to use; including driver domains, normal domains, stub domains, and so on. However, these example policies are not heavily tested; they should be used at your own risk.

The basic steps to use FLASK's example policies are as follows:

  • Build Xen with XSM enabled
  • Build the example policy
  • Add the appropriate "labels" to the VM config files, using seclabel=[$foo], based on their role.

For more information, please see Xen_Security_Modules_:_XSM-FLASK

Also See