Network Configuration Examples (Xen 4.1+)

Overview

When using the xl toolstack the host networking configuration is not configured by the toolstack but rather administrators are required to setup an appropriate network configuration using the tools provided by their host distribution.

From Xen 4.1 onwards the xend toolstack will only reconfigure the host network stack if the network stack does not appear to have been configured already (e.g. no bridges currently exist). This change allows administrators who wish to configure the network stack themselves to do so by default while preserving the existing behaviour for those who do not. TO force xend to never try and reconfigure networking edit /etc/xen/xend-config.sxp and remove any (network-script ...) options.

Other toolstack's requirements may differ. e.g. XCP contains its own mechanisms for configuring networking which should be used instead.

It is recommended to use the xenbrN naming convention for maximum compatibility.

Distribution Network Configuration Examples

The following sections contain examples of common network configurations for various Linux distributions.

A correctly configured host network should have a bridge device (or "shared physical device" in libvirt terms), to which guests can be attached and have full LAN access. This can be seen in the output of the brctl show command.

 bridge name     bridge id               STP enabled     interfaces
 xenbr0          8000.000e0cb30550       yes             eth0

Note: Your system may be configured several bridges. e.g. libvirt will create a bridge called virbr0.

Example Debian-style bridge configuration (e.g. Debian, Ubuntu)

Under Debian (and its derivative distributions) the basic network configuration is managed by the ifupdown tool and configured via the /etc/network/interfaces configuration file which is described in the interfaces(5) man page. The Linux bridge configuration tools are supplied in the bridge-utils package which integrates support into /etc/network/interfaces as described in the bridge-utils-interfaces(5) man page.

The following examples show /etc/network/interfaces stanzas which setup common network configurations. Substitute xenbr0 and eth0 as necessary. In most cases you will also want to include an auto xenbr0 line to cause the bridge to be brought up on boot.

The configuration of the bridge device should replace the configuration of the underlying interface. i.e., remove the stanza for eth0.

Example 1: A single bridged network using eth0 configured with a local IP address via DHCP

iface xenbr0 inet dhcp
	bridge_ports eth0

Example 2: A single bridged network using eth0 configured with a static local IP address

iface xenbr0 inet static
        bridge_ports eth0
        address 192.168.1.2
        broadcast 192.168.1.255
        netmask 255.255.255.0
        gateway 192.168.1.1

Example 3: A single bridged network using eth0 with no local IP address

iface xenbr0 inet manual
	bridge_ports eth0

Example 4: An internal bridge with no external connectivity. Note that $IFACE here can be entered literally, it is substituted automatically by ifupdown

iface xenbr0 inet manual
	pre-up brctl addbr $IFACE
	up ip link set $IFACE up
	post-down brctl delbr $IFACE
	down ip link set $IFACE down

Some other useful options to use in any stanza in a virtualised environment are:

	bridge_stp off		# disable Spanning Tree Protocol
	bridge_waitport 0	# no delay before a port becomes available
	bridge_fd 0		# no forwarding delay

Further information is available in the Debian Wiki.

Example Debian-style internal dummy network bridge configuration (e.g. Debian, Ubuntu)

The dummy virtual network devices have all the functionality of physical network interfaces are used to create private networks that do not have access to a physical network. These serve to connect DomUs to each other without connecting them to the outside world.

Example 1: A internal bridged network using dummy interface: /etc/network/interfaces

auto dummy0
iface dummy0 inet manual
         pre-up ifconfig $IFACE up
         post-down ifconfig $IFACE down

auto xenbrdummy
iface xenbrdummy inet manual
         bridge_ports dummy0
         bridge_maxwait 0
         bridge_stp off

Red Hat-style bridge configuration (e.g. RHEL, Fedora, CentOS)

As of the time of writing (Fedora 12), NetworkManager still does not support bridging, so it is necessary to use "classic" network initscripts for the bridge, and to explicitly mark them as independent from NetworkManager (the "NM_CONTROLLED=no" lines in the scripts below).

If desired you can completely disable the NetworkManager by running the following commands:

Example 1: Disabling NetworkManager

# chkconfig NetworkManager off
# chkconfig network on
# service NetworkManager stop
# service network start

In order to create a bridged network configuration on a Red Hat-style system it is necessary to create two ifcfg configuration files under /etc/sysconfig/network-scripts//. The first configures the phyical network device to be placed on a specific bridge. The second (see Example 2 onwards) configures the bridge itself and includes any necessary IP address configuration.

All ifcfg files are case sensitive. In particular Bridge must be written exactly as shown.

Example 1: A physical network device on a bridge: /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
HWADDR=00:16:76:D6:C9:45
ONBOOT=yes
BRIDGE=xenbr0
NM_CONTROLLED=no

Change the HWADDR to match your actual NIC's address. This configuration is the counterpart used by all of the following xenbr0 examples.

Example 2: A single bridged network configured with a local IP address via DHCP: /etc/sysconfig/network-scripts/ifcfg-xenbr0

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 3: A single bridged network configured with a static local IP address: /etc/sysconfig/network-scripts/ifcfg-xenbr0

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=static
BROADCAST=192.168.1.255
IPADDR=192.168.1.2
NETMASK=255.255.255.0
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 4: A single bridged network with no local IP address: /etc/sysconfig/network-scripts/ifcfg-xenbr0

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 5: An internal bridge with no external connectivity: /etc/sysconfig/network-scripts/ifcfg-xenbr0 An internal bridge can be created as per Example 4 but omitting the eth0 configuration show in example 1.

Some other useful options to use in any stanza are:

MTU=9000                        # Configure Jumbo frames

}

After changing this restart networking by running:

# service network restart

For performance and security reasons it is highly recommended to disable netfilter on the bridge by adding the following to /etc/sysctl.conf. See Fedora Bug #512206 for more details.

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

The run

# sysctl -p /etc/sysctl.conf

It is recommended to do this for performance and security reasons. See Fedora bug #512206.

Alternatively you can configure iptables to allow all traffic to be forwarded across the bridge:

echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged
lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged
service libvirtd reload

Further information is available in the libvirt wiki.