Network Configuration Examples (Xen 4.1+): Difference between revisions

From Xen
Jump to navigationJump to search
mNo edit summary
(Replace <pre>...</pre> blocks with <syntaxhighlight lang="sh"> ... </syntaxhighlight>)
 
(11 intermediate revisions by 5 users not shown)
Line 3: Line 3:
When using the <code>xl</code> toolstack the host networking configuration is not configured by the toolstack but rather administrators are required to setup an appropriate network configuration using the tools provided by their host distribution.
When using the <code>xl</code> toolstack the host networking configuration is not configured by the toolstack but rather administrators are required to setup an appropriate network configuration using the tools provided by their host distribution.


There are two main styles of network setup for a Xen host, ''bridged'' and ''routed''. The default and most common is ''bridged''. See [[Xen Networking]] for a general discussion of what each of these options mean.
There are three main styles of network setup for a Xen host, ''bridged'', ''routed'' and ''nat''. The default and most common is ''bridged''. From Xen 4.3 onwards, ''openvswitch'' is also supported. See [[Xen Networking]] for a general discussion of what each of these options mean.


From Xen 4.1 onwards the <code>xend</code> toolstacks <code>network-bridge</code> script will only reconfigure the host network stack if the network stack does not appear to have been configured already (e.g. no bridges currently exist). This change allows administrators who wish to configure the network stack themselves to do so by default while preserving the existing behaviour for those who do not. Other <code>network-*</code> scripts will still unconditionally reconfigure networking when called by <code>xend</code>. To force <code>xend</code> to never try and reconfigure networking edit ''/etc/xen/xend-config.sxp'' and remove any ''(network-script ...)'' options.
From Xen 4.1 onwards the <code>xend</code> toolstacks <code>network-bridge</code> script will only reconfigure the host network stack if the network stack does not appear to have been configured already (e.g. no bridges currently exist). This change allows administrators who wish to configure the network stack themselves to do so by default while preserving the existing behaviour for those who do not. Other <code>network-*</code> scripts will still unconditionally reconfigure networking when called by <code>xend</code>. To force <code>xend</code> to never try and reconfigure networking edit ''/etc/xen/xend-config.sxp'' and remove any ''(network-script ...)'' options.
Line 19: Line 19:
A host with correctly configured bridged networking should have a bridge device (or "shared physical device" in libvirt terms), to which guests can be attached and have full LAN access. This can be seen in the output of the <code>brctl show</code> command.
A host with correctly configured bridged networking should have a bridge device (or "shared physical device" in libvirt terms), to which guests can be attached and have full LAN access. This can be seen in the output of the <code>brctl show</code> command.


<syntaxhighlight lang="sh">
<pre>
bridge name bridge id STP enabled interfaces
bridge name bridge id STP enabled interfaces
xenbr0 8000.000e0cb30550 yes eth0
xenbr0 8000.000e0cb30550 yes eth0
</syntaxhighlight>
</pre>


'''Note''': Your system may be configured several bridges. e.g. <code>libvirt</code> will create a bridge called <code>virbr0</code>.
'''Note''': Your system may be configured several bridges. e.g. <code>libvirt</code> will create a bridge called <code>virbr0</code>.
Line 30: Line 30:
In addition to the per distribution examples below it is highly recommended for performance and security reasons that netfilter is disabled on all bridges by adding the following to <code>/etc/sysctl.conf</code>. See Fedora Bug [https://bugzilla.redhat.com/show_bug.cgi?id=512206 #512206] for more details.
In addition to the per distribution examples below it is highly recommended for performance and security reasons that netfilter is disabled on all bridges by adding the following to <code>/etc/sysctl.conf</code>. See Fedora Bug [https://bugzilla.redhat.com/show_bug.cgi?id=512206 #512206] for more details.


<syntaxhighlight lang="sh">
<pre>
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
net.bridge.bridge-nf-call-arptables = 0
</syntaxhighlight>
</pre>


Then run, as root:
Then run, as root:
<syntaxhighlight lang="sh">
<pre>
# sysctl -p /etc/sysctl.conf
# sysctl -p /etc/sysctl.conf
</syntaxhighlight>
</pre>


Alternatively you can configure <code>iptables</code> to allow all traffic to be forwarded across the bridge by adding the following rule:
Alternatively you can configure <code>iptables</code> to allow all traffic to be forwarded across the bridge by adding the following rule:
<syntaxhighlight lang="sh">
<pre>
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT
</syntaxhighlight>
</pre>
The manner in which this can be achieved is distro specific.
The manner in which this can be achieved is distro specific.


Line 53: Line 53:
The following examples show <code>/etc/network/interfaces</code> stanzas which setup common network configurations. Substitute <code>xenbr0</code> and <code>eth0</code> as necessary. In most cases you will also want to include an <code>auto xenbr0</code> line to cause the bridge to be brought up on boot.
The following examples show <code>/etc/network/interfaces</code> stanzas which setup common network configurations. Substitute <code>xenbr0</code> and <code>eth0</code> as necessary. In most cases you will also want to include an <code>auto xenbr0</code> line to cause the bridge to be brought up on boot.


'''IMPORTANT NOTE !'''
Note! The IP configuration of the bridge device should replace the IP configuration of the underlying interface, i.e. remove the IP settings from eth0 and move them to the bridge interface. eth0 will function purely as the physical uplink from the bridge so it can't have any IP (L3) settings on it!
The IP configuration of the bridge device should replace the IP configuration
of the underlying interface, '''i.e. remove the IP settings from eth0''' and move
them to the bridge interface. <br/>
eth0 will function purely as the physical uplink from the bridge so it can't
have any IP (L3) settings on it!


''Example 1: A single bridged network using eth0 configured with a local IP address via DHCP''
''Example 1: A single bridged network using eth0 configured with a local IP address via DHCP''
<syntaxhighlight lang="sh">
<pre>
iface eth0 inet manual

iface xenbr0 inet dhcp
iface xenbr0 inet dhcp
bridge_ports eth0
bridge_ports eth0
</syntaxhighlight>
</pre>




''Example 2: A single bridged network using eth0 configured with a static local IP address''
''Example 2: A single bridged network using eth0 configured with a static local IP address''
<syntaxhighlight lang="sh">
<pre>
iface eth0 inet manual

iface xenbr0 inet static
iface xenbr0 inet static
bridge_ports eth0
bridge_ports eth0
Line 70: Line 79:
netmask 255.255.255.0
netmask 255.255.255.0
gateway 192.168.1.1
gateway 192.168.1.1
</syntaxhighlight>
</pre>




''Example 3: A single bridged network using eth0 with no local IP address''
''Example 3: A single bridged network using eth0 with no local IP address''
<syntaxhighlight lang="sh">
<pre>
iface eth0 inet manual

iface xenbr0 inet manual
iface xenbr0 inet manual
bridge_ports eth0
bridge_ports eth0
</syntaxhighlight>
</pre>




''Example 4: An internal bridge with no external connectivity. Note that <code>$IFACE</code> here can be entered literally, it is substituted automatically by <code>ifupdown</code>''
''Example 4: An internal bridge with no external connectivity. Note that <code>$IFACE</code> here can be entered literally, it is substituted automatically by <code>ifupdown</code>''
<syntaxhighlight lang="sh">
<pre>
iface xenbr0 inet manual
iface xenbr0 inet manual
pre-up brctl addbr $IFACE
pre-up brctl addbr $IFACE
Line 87: Line 98:
post-down brctl delbr $IFACE
post-down brctl delbr $IFACE
down ip link set $IFACE down
down ip link set $IFACE down
</syntaxhighlight>
</pre>


''Example 5: A bridge on VLAN #2 of device eth0'':
<syntaxhighlight lang="sh">
iface xenbr0.2 inet dhcp
bridge_ports eth0.2
</syntaxhighlight>
The syntax for supporting VLANs in this way is described in the <code>vlan-interfaces(5)</code> man page. VLAN support requires the <code>vconfig</code> tool from the <code>vlan</code> package is installed.


Some other useful options to use in any stanza in a virtualised environment are:
Some other useful options to use in any stanza in a virtualised environment are:
<syntaxhighlight lang="sh">
<pre>
bridge_stp off # disable Spanning Tree Protocol
bridge_stp off # disable Spanning Tree Protocol
bridge_waitport 0 # no delay before a port becomes available
bridge_waitport 0 # no delay before a port becomes available
bridge_fd 0 # no forwarding delay
bridge_fd 0 # no forwarding delay
</syntaxhighlight>
</pre>


For some environments, when using openvswitch, you may need to add the following line:
<syntaxhighlight lang="sh">
allow-hotplug xenbr0
</syntaxhighlight>


Further information is available in the [http://wiki.debian.org/BridgeNetworkConnections Debian Wiki].
Further information about configuring bridging on Debian-like systems is available in the [http://wiki.debian.org/BridgeNetworkConnections Debian Wiki].

==== Example Debian-style internal dummy network bridge configuration (e.g. Debian, Ubuntu) ====

The dummy virtual network devices have all the functionality of physical network interfaces are used to create private networks that do not have access to a physical network. These serve to connect [[DomUs]] to each other without connecting them to the outside world.

''Example 1: A internal bridged network using dummy interface:'' <code>/etc/network/interfaces</code>
<pre>
auto dummy0
iface dummy0 inet manual
pre-up ifconfig $IFACE up
post-down ifconfig $IFACE down

auto xenbrdummy
iface xenbrdummy inet manual
bridge_ports dummy0
bridge_maxwait 0
bridge_stp off
</pre>



==== Red Hat-style bridge configuration (e.g. RHEL, Fedora, CentOS) ====
==== Red Hat-style bridge configuration (e.g. RHEL, Fedora, CentOS) ====


As of the time of writing (Fedora 12), [[NetworkManager]] still does not support bridging, so it is necessary to use "classic" network initscripts for the bridge, and to explicitly mark them as independent from [[NetworkManager]] (the "NM_CONTROLLED=no" lines in the scripts below).
As of the time of writing (Fedora 12), NetworkManager still does not support bridging, so it is necessary to use "classic" network initscripts for the bridge, and to explicitly mark them as independent from NetworkManager (the "<code>NM_CONTROLLED=no</code>" lines in the scripts below).


If desired you can completely disable the [[NetworkManager]] by running the following commands:
If desired you can completely disable the NetworkManager by running the following commands:


''Example 1: Disabling [[NetworkManager]]''
''Example 1: Disabling NetworkManager''
<syntaxhighlight lang="sh">
<pre>
# chkconfig NetworkManager off
# chkconfig NetworkManager off
# chkconfig network on
# chkconfig network on
# service NetworkManager stop
# service NetworkManager stop
# service network start
# service network start
</syntaxhighlight>
</pre>


In order to create a bridged network configuration on a Red Hat-style system it is necessary to create two <code>ifcfg</code> configuration files under <code>/etc/sysconfig/network-scripts//</code>. The first configures the phyical network device to be placed on a specific bridge. The second (see ''Example 2'' onwards) configures the bridge itself and includes any necessary IP address configuration.
In order to create a bridged network configuration on a Red Hat-style system it is necessary to create two <code>ifcfg</code> configuration files under <code>/etc/sysconfig/network-scripts//</code>. The first configures the physical network device to be placed on a specific bridge. The second (see ''Example 2'' onwards) configures the bridge itself and includes any necessary IP address configuration.


All <code>ifcfg</code> files are case sensitive. In particular <code>Bridge</code> must be written exactly as shown.
All <code>ifcfg</code> files are case sensitive. In particular <code>Bridge</code> must be written exactly as shown.


''Example 1: A physical network device on a bridge: /etc/sysconfig/network-scripts/ifcfg-eth0''
''Example 1: A physical network device on a bridge: /etc/sysconfig/network-scripts/ifcfg-eth0''
<syntaxhighlight lang="sh">
<pre>
DEVICE=eth0
DEVICE=eth0
HWADDR=00:16:76:D6:C9:45
HWADDR=00:16:76:D6:C9:45
Line 144: Line 146:
BRIDGE=xenbr0
BRIDGE=xenbr0
NM_CONTROLLED=no
NM_CONTROLLED=no
</syntaxhighlight>
</pre>


Change the <code>HWADDR</code> to match your actual NIC's address. This configuration is the counterpart used by all of the following <code>xenbr0</code> examples.
Change the <code>HWADDR</code> to match your actual NIC's address. This configuration is the counterpart used by all of the following <code>xenbr0</code> examples.


''Example 2: A single bridged network configured with a local IP address via DHCP: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
''Example 2: A single bridged network configured with a local IP address via DHCP: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
<syntaxhighlight lang="sh">
<pre>
DEVICE=xenbr0
DEVICE=xenbr0
TYPE=Bridge
TYPE=Bridge
Line 156: Line 158:
DELAY=0
DELAY=0
NM_CONTROLLED=no
NM_CONTROLLED=no
</syntaxhighlight>
</pre>


''Example 3: A single bridged network configured with a static local IP address: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
''Example 3: A single bridged network configured with a static local IP address: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
<syntaxhighlight lang="sh">
<pre>
DEVICE=xenbr0
DEVICE=xenbr0
TYPE=Bridge
TYPE=Bridge
Line 169: Line 171:
DELAY=0
DELAY=0
NM_CONTROLLED=no
NM_CONTROLLED=no
</syntaxhighlight>
</pre>


''Example 4: A single bridged network with no local IP address: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
''Example 4: A single bridged network with no local IP address: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
<syntaxhighlight lang="sh">
<pre>
DEVICE=xenbr0
DEVICE=xenbr0
TYPE=Bridge
TYPE=Bridge
Line 179: Line 181:
DELAY=0
DELAY=0
NM_CONTROLLED=no
NM_CONTROLLED=no
</syntaxhighlight>
</pre>


''Example 5: An internal bridge with no external connectivity: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
''Example 5: An internal bridge with no external connectivity: <code>/etc/sysconfig/network-scripts/ifcfg-xenbr0</code>''
Line 186: Line 188:


Some other useful options to use in any stanza are:
Some other useful options to use in any stanza are:
<syntaxhighlight lang="sh">
<pre>
MTU=9000 # Configure Jumbo frames
MTU=9000 # Configure Jumbo frames
</syntaxhighlight>
</pre>


After changing this restart networking by running:
After changing this restart networking by running:
<syntaxhighlight lang="sh">
<pre>
# service network restart
# service network restart
</syntaxhighlight>
</pre>


As discussed above it is recommended to disable netfilter for bridges. However you can alternatively you can configure <code>iptables</code> to allow all traffic to be forwarded across the bridge as follows:
As discussed above it is recommended to disable netfilter for bridges. However you can alternatively you can configure <code>iptables</code> to allow all traffic to be forwarded across the bridge as follows:


<syntaxhighlight lang="sh">
<pre>
echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged
echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged
lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged
lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged
service libvirtd reload
service libvirtd reload
</syntaxhighlight>
</pre>


Further information is available in the [http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging libvirt wiki].
Further information is available in the [http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging libvirt wiki].

=== Openvswitch ===

The configuration for openvswitch is persistent across reboots. Supposing that you want to create a bridge called 'xenbr0' and attach it to physical nic 'eth0', after installation you can use the following commands:
<syntaxhighlight lang="sh">
ovs-vsctl add-br xenbr0
ovs-vsctl add-port xenbr0 eth0
</syntaxhighlight>

<code>xenbr0</code> will still need to be set up to acquire an IP address just as Linux bridging does; see above for examples on how to set this up for the various distributions.


=== Routing ===
=== Routing ===


In order to configure routed networking on a host it is necessary to enable IP forwarding on the host and proxy [http://en.wikipedia.org/wiki/Address_Resolution_Protocol ARP] for the physical devices which will be handling traffic.
In order to configure routed networking on a host it is necessary to enable IP forwarding on the host.

In addition unless you control the routing tables of the upstream gateway it will also be necessary to enable [http://en.wikipedia.org/wiki/Proxy_ARP proxy ARP] for the physical devices which will be handling traffic. This will cause the domain 0 kernel to reply to [http://en.wikipedia.org/wiki/ARP ARP] requests on behalf of the guests in order to cause traffic for those guests to be routed to domain 0 such that they can be forwarded to the domain.


In these examples we assume that <code>eth0</code> is the physical interface used to route traffic.
In these examples we assume that <code>eth0</code> is the physical interface used to route traffic.
Line 215: Line 229:
These options are controlled via the sysctl interface and are documented in [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=Documentation/networking/ip-sysctl.txt;hb=HEAD networking/ip-sysctl.txt] in the Linux kernel source.
These options are controlled via the sysctl interface and are documented in [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=Documentation/networking/ip-sysctl.txt;hb=HEAD networking/ip-sysctl.txt] in the Linux kernel source.


To enable these options edit the file <code>/etc/sysctl.conf</code> and add or uncomment the following lines:
To enable these options edit the file <code>/etc/sysctl.conf</code> and add or uncomment the following lines:
<syntaxhighlight lang="sh">
<pre>
net.ipv4.ip_forward = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.eth0.proxy_arp = 1
net.ipv4.conf.eth0.proxy_arp = 1
</syntaxhighlight>
</pre>


To make this change take effect immediately run, as root:
To make this change take effect immediately run, as root:
<syntaxhighlight lang="sh">
<pre>
# sysctl -p /etc/sysctl.conf
# sysctl -p /etc/sysctl.conf
</syntaxhighlight>
</pre>

=== Network Address Translation (NAT) ===

Enabling NAT is similar to enabling routing in that you must enable IP forwarding. You also need to configure Masquerading in the host firewall using [http://www.netfilter.org/projects/iptables/ iptables].

In addition you may optionally wish to configure a DHCP server to provide IP addresses to the guests on the private network. This is not required and you can also just use static IP addresses configured in the guests themselves.

==== Enabling NAT (All Distributions) ====

The IP forwarding option is controlled via the sysctl interface (documented in [http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=Documentation/networking/ip-sysctl.txt;hb=HEAD networking/ip-sysctl.txt] in the Linux kernel source).

To enable edit the file <code>/etc/sysctl.conf</code> and add or uncomment the following line:
<syntaxhighlight lang="sh">
net.ipv4.ip_forward = 1
</syntaxhighlight>

To make this change take effect immediately run, as root:
<syntaxhighlight lang="sh">
# sysctl -p /etc/sysctl.conf
</syntaxhighlight>

To enable masquerading via the device <code>eth0</code> add the following rule to iptables:
<syntaxhighlight lang="sh">
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
</syntaxhighlight>

Making this change permanent is distro specific (TBD: HOW?)

TBD: Setting up a DHCP controller.

== Distribution Specific Resources ==

In general most host level networking configuration is done using the tools provided by the domain 0 Operating System and is not in any way specific to Xen. This section links to various resources provided by distributions.

=== Debian & Ubuntu ===

* http://wiki.debian.org/NetworkConfiguration
* http://wiki.debian.org/BridgeNetworkConnections

=== Fedora ===

* http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging

=== Gentoo ===

* https://wiki.gentoo.org/wiki/Network_bridge


[[Category:Xen]]
[[Category:Xen]]

Latest revision as of 13:00, 10 August 2016

Overview

When using the xl toolstack the host networking configuration is not configured by the toolstack but rather administrators are required to setup an appropriate network configuration using the tools provided by their host distribution.

There are three main styles of network setup for a Xen host, bridged, routed and nat. The default and most common is bridged. From Xen 4.3 onwards, openvswitch is also supported. See Xen Networking for a general discussion of what each of these options mean.

From Xen 4.1 onwards the xend toolstacks network-bridge script will only reconfigure the host network stack if the network stack does not appear to have been configured already (e.g. no bridges currently exist). This change allows administrators who wish to configure the network stack themselves to do so by default while preserving the existing behaviour for those who do not. Other network-* scripts will still unconditionally reconfigure networking when called by xend. To force xend to never try and reconfigure networking edit /etc/xen/xend-config.sxp and remove any (network-script ...) options.

Other toolstack's requirements may differ. e.g. XCP contains its own mechanisms for configuring networking which should be used instead.

When using bridging it is recommended to use the xenbrN naming convention for maximum compatibility.

Distribution Network Configuration Examples

The following sections contain examples of common network configurations for various Linux distributions.

Bridging

A host with correctly configured bridged networking should have a bridge device (or "shared physical device" in libvirt terms), to which guests can be attached and have full LAN access. This can be seen in the output of the brctl show command. 

 bridge name     bridge id               STP enabled     interfaces
 xenbr0          8000.000e0cb30550       yes             eth0

Note: Your system may be configured several bridges. e.g. libvirt will create a bridge called virbr0.

Disable Netfilter on Bridges (All Distributions)

In addition to the per distribution examples below it is highly recommended for performance and security reasons that netfilter is disabled on all bridges by adding the following to /etc/sysctl.conf. See Fedora Bug #512206 for more details. 

net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

Then run, as root: 

# sysctl -p /etc/sysctl.conf

Alternatively you can configure iptables to allow all traffic to be forwarded across the bridge by adding the following rule: 

-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

The manner in which this can be achieved is distro specific.

Example Debian-style bridge configuration (e.g. Debian, Ubuntu)

Under Debian (and its derivative distributions) the basic network configuration is managed by the ifupdown tool and configured via the /etc/network/interfaces configuration file which is described in the interfaces(5) man page. The Linux bridge configuration tools are supplied in the bridge-utils package which integrates support into /etc/network/interfaces as described in the bridge-utils-interfaces(5) man page.

The following examples show /etc/network/interfaces stanzas which setup common network configurations. Substitute xenbr0 and eth0 as necessary. In most cases you will also want to include an auto xenbr0 line to cause the bridge to be brought up on boot. 

IMPORTANT NOTE !
The IP configuration of the bridge device should replace the IP configuration 
of the underlying interface, i.e. remove the IP settings from eth0 and move
them to the bridge interface. 

eth0 will function purely as the physical uplink from the bridge so it can't 
have any IP (L3) settings on it!

Example 1: A single bridged network using eth0 configured with a local IP address via DHCP 

iface eth0 inet manual

iface xenbr0 inet dhcp
	bridge_ports eth0


Example 2: A single bridged network using eth0 configured with a static local IP address 

iface eth0 inet manual

iface xenbr0 inet static
        bridge_ports eth0
        address 192.168.1.2
        broadcast 192.168.1.255
        netmask 255.255.255.0
        gateway 192.168.1.1


Example 3: A single bridged network using eth0 with no local IP address 

iface eth0 inet manual

iface xenbr0 inet manual
	bridge_ports eth0


Example 4: An internal bridge with no external connectivity. Note that $IFACE here can be entered literally, it is substituted automatically by ifupdown 

iface xenbr0 inet manual
	pre-up brctl addbr $IFACE
	up ip link set $IFACE up
	post-down brctl delbr $IFACE
	down ip link set $IFACE down

Example 5: A bridge on VLAN #2 of device eth0: 

iface xenbr0.2 inet dhcp
	bridge_ports eth0.2

The syntax for supporting VLANs in this way is described in the vlan-interfaces(5) man page. VLAN support requires the vconfig tool from the vlan package is installed.

Some other useful options to use in any stanza in a virtualised environment are: 

	bridge_stp off		# disable Spanning Tree Protocol
	bridge_waitport 0	# no delay before a port becomes available
	bridge_fd 0		# no forwarding delay

For some environments, when using openvswitch, you may need to add the following line: 

allow-hotplug xenbr0

Further information about configuring bridging on Debian-like systems is available in the Debian Wiki.

Red Hat-style bridge configuration (e.g. RHEL, Fedora, CentOS)

As of the time of writing (Fedora 12), NetworkManager still does not support bridging, so it is necessary to use "classic" network initscripts for the bridge, and to explicitly mark them as independent from NetworkManager (the "NM_CONTROLLED=no" lines in the scripts below).

If desired you can completely disable the NetworkManager by running the following commands:

Example 1: Disabling NetworkManager 

# chkconfig NetworkManager off
# chkconfig network on
# service NetworkManager stop
# service network start

In order to create a bridged network configuration on a Red Hat-style system it is necessary to create two ifcfg configuration files under /etc/sysconfig/network-scripts//. The first configures the physical network device to be placed on a specific bridge. The second (see Example 2 onwards) configures the bridge itself and includes any necessary IP address configuration.

All ifcfg files are case sensitive. In particular Bridge must be written exactly as shown.

Example 1: A physical network device on a bridge: /etc/sysconfig/network-scripts/ifcfg-eth0 

DEVICE=eth0
HWADDR=00:16:76:D6:C9:45
ONBOOT=yes
BRIDGE=xenbr0
NM_CONTROLLED=no

Change the HWADDR to match your actual NIC's address. This configuration is the counterpart used by all of the following xenbr0 examples.

Example 2: A single bridged network configured with a local IP address via DHCP: /etc/sysconfig/network-scripts/ifcfg-xenbr0 

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 3: A single bridged network configured with a static local IP address: /etc/sysconfig/network-scripts/ifcfg-xenbr0 

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=static
BROADCAST=192.168.1.255
IPADDR=192.168.1.2
NETMASK=255.255.255.0
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 4: A single bridged network with no local IP address: /etc/sysconfig/network-scripts/ifcfg-xenbr0 

DEVICE=xenbr0
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DELAY=0
NM_CONTROLLED=no

Example 5: An internal bridge with no external connectivity: /etc/sysconfig/network-scripts/ifcfg-xenbr0

An internal bridge can be created as per Example 4 but omitting the eth0 configuration shown in example 1.

Some other useful options to use in any stanza are: 

MTU=9000                        # Configure Jumbo frames

After changing this restart networking by running: 

# service network restart

As discussed above it is recommended to disable netfilter for bridges. However you can alternatively you can configure iptables to allow all traffic to be forwarded across the bridge as follows: 

echo "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT" > /etc/sysconfig/iptables-forward-bridged
lokkit --custom-rules=ipv4:filter:/etc/sysconfig/iptables-forward-bridged
service libvirtd reload

Further information is available in the libvirt wiki.

Openvswitch

The configuration for openvswitch is persistent across reboots. Supposing that you want to create a bridge called 'xenbr0' and attach it to physical nic 'eth0', after installation you can use the following commands: 

ovs-vsctl add-br xenbr0
ovs-vsctl add-port xenbr0 eth0

xenbr0 will still need to be set up to acquire an IP address just as Linux bridging does; see above for examples on how to set this up for the various distributions.

Routing

In order to configure routed networking on a host it is necessary to enable IP forwarding on the host.

In addition unless you control the routing tables of the upstream gateway it will also be necessary to enable proxy ARP for the physical devices which will be handling traffic. This will cause the domain 0 kernel to reply to ARP requests on behalf of the guests in order to cause traffic for those guests to be routed to domain 0 such that they can be forwarded to the domain.

In these examples we assume that eth0 is the physical interface used to route traffic.

Enabling Routing (All Distributions)

These options are controlled via the sysctl interface and are documented in networking/ip-sysctl.txt in the Linux kernel source.

To enable these options edit the file /etc/sysctl.conf and add or uncomment the following lines: 

net.ipv4.ip_forward = 1
net.ipv4.conf.eth0.proxy_arp = 1

To make this change take effect immediately run, as root: 

# sysctl -p /etc/sysctl.conf

Network Address Translation (NAT)

Enabling NAT is similar to enabling routing in that you must enable IP forwarding. You also need to configure Masquerading in the host firewall using iptables.

In addition you may optionally wish to configure a DHCP server to provide IP addresses to the guests on the private network. This is not required and you can also just use static IP addresses configured in the guests themselves.

Enabling NAT (All Distributions)

The IP forwarding option is controlled via the sysctl interface (documented in networking/ip-sysctl.txt in the Linux kernel source).

To enable edit the file /etc/sysctl.conf and add or uncomment the following line: 

net.ipv4.ip_forward = 1

To make this change take effect immediately run, as root: 

# sysctl -p /etc/sysctl.conf

To enable masquerading via the device eth0 add the following rule to iptables: 

# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Making this change permanent is distro specific (TBD: HOW?)

TBD: Setting up a DHCP controller.

Distribution Specific Resources

In general most host level networking configuration is done using the tools provided by the domain 0 Operating System and is not in any way specific to Xen. This section links to various resources provided by distributions.

Debian & Ubuntu

Fedora

Gentoo

Retrieved from "https://wiki.xenproject.org/index.php?title=Network_Configuration_Examples_(Xen_4.1%2B)&oldid=17199"