Xen FAQ Networking

From Xen
Revision as of 11:02, 30 July 2012 by OliverChick (talk | contribs) (Removed line breaks that prevented a Section header from being correctly parsed.)


Icon todo.png Needs Review

This page is very out-of-date. Many of the issues described on this page have been resolved.


Contents

Networking Issues

What is veth, vif or xenbr0?

You should read XenNetworking http://wiki.xen.org/wiki/XenNetworking

Why can't I ssh into or ping a newly created domain?

In the default configuration we rely on the Linux bridge-utils in domain 0 to set up virtual networking. After you've created a new domain (e.g., domain 1) you should be able to run ifconfig in domain 0 and see an interface with a name like vif1.0; you should also be able to check that bridging is working by typing brctl show xen-br0. Finally, you can check the IP confiuration in the new domain by logging into it via the console (xm console) and running standard tools such as ifconfig and route.

TCP and UDP checksum errors, ping but nothing else, ipsec tunnels don't form, DNAT translation doesn't work

Try running the following command in each domU:


ethtool -K eth0 tx off

This command disable TX checksumming

To check checksum problems, you can use tcpdump:


tcpdump -vv -n -i eth0

Read http://lists.xensource.com/archives/html/xen-users/2006-04/msg00032.html

This may or may not work.

Patch for network-bridge

Xen and Shorewall

There is a document about configuring Shorewall in Dom0 at http://www.shorewall.net/Xen.html

http://www1.shorewall.net/XenMyWay.html can be useful also.

Too many vethX and vif0.X

By default, 8 vethX and vif0.X are created. These interfaces are "cheap" but clutter list of interfaces. If you want to allocate only the necessary number, pass netloop.nloopbacks=NUMBER parameter to kernel command line.

I can't use more than 3 network interfaces in domU

This is a limitation in Xen 3. Xen 3.1 supports 8 network interfaces.

Bridging

Which Mechanism is used by Xen bridging to handle packets coming from various VMs to forward them to their destination

Nothing. Xen by itself does not handle bridge. dom0 OS does that. On Linux dom0 : http://www.linuxfoundation.org/en/Net:Bridge On opensolaris dom0: http://opensolaris.org/os/project/crossbow/

IP Determination

I want to know the IP of a running VM in XEN. Is there any way to have this without login to that VM?

Yes. First, you need to find the MAC address of the domU. This can be done by running:

xl network-list <domU name>

which should produce an output similar to:

Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path 0 0 00:16:3E:F7:D6:E7 0 4
6 16238/16237 /local/domain/0/backend/vif/163/0

The domU's MAC address is 00:16:3E:F7:D6:E7

You now need to snoop the bridge for domU's MAC. For example:

# tcpdump -n -i eth0 ether src 00:16:3e:f7:d6:e7 tcpdump: verbose output suppressed, use
-v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
15:54:56.419482 IP 10.0.0.10 > 10.0.0.1: ICMP echo reply, id 5443, seq 1, length 64 15:54:57.422349
IP 10.0.0.10 > 10.0.0.1: ICMP echo reply, id 5443, seq 2, length 64


Then you know that domU has IP address 10.0.0.10.

NAT

I managed to configure NAT on dom0 but this does not work properly. Outgoing traffic from domU is seen with the original domU ip address instead of the dom0 ip address and the requests can't get back to the domU.

I figured out MASQUERADING was not set.

The following rule needs to be set:


iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE


SSL/VPN

Why can't I use openvpn with a xen guest? I can't load the tun module

From openvpn perspective, the requirements for openvpn on xen domU is the same as openvpn on native Linux. If you can't load the tun module, then you need to get a kernel that supports it.

The easiest way is to use a distro that supports it. For example, I'm using RHEL/Centos 5.3 domU, loaded with pygrub, and they can run openvpn just fine.

Another alternative is compile your own kernel with tun/tap support.

I want to ask how to mount one storage device to 2 guests? When I try to create vm handle everything is fine I can create one vdi and vbds for every guest. When I start first machine everything is ok, but when I try to start second one it says that

ERROR: 2 INTERNAL_ERROR Device 2051 (vbd) could not be connected. Device /dev/mapper/test_vg-test64_454 is mounted in a guest domain,
and so cannot be mounted now.


Ideas how I can share one device to two or more virtual machines? I don't want network solution like nfs,iscsi ... etc but instead use ocfs2. How I can set mode to w! ?

First of all, you DO realize that sharing a block device without some kind of cluster file system could lead to data corruption? If you want to share it anyway, you can try changing the mode to "r"(for read only) or "w!" (to force read-write multiple mount).

in your domU.cfg:

 
'phy:/dev/data/bla-disk,sda1,w' => 'phy:/dev/data/bla-disk,sda1,r'


I'm looking for a way to monitor network activities of processes in Guest OS. I want to get a list of Guest OS processes that open TCP connections to other machines (like "lsof" command).

If you're thinking about doing on from dom0, that's not possible. You need something that runs on domU for that, possibly by using snmpd and extending it to run "netstat -anp --tcp". Other host (including dom0) can then collect the information using snmp.

Also, have a look at Versiera, it provides what you are looking for including, user IDs, inbound/outbound communications, IPv4, IPV6, etc. There are many more capabilities. Versiera is not open-source, but the Internet self-manage service is free.

I am attempting to gather stats on usage of the "metal", by which I mean the physical host's hardware. I would like to know the CPU, IO, and network stats for the hardware.

All DomU's IO passes through Dom0. there you can measure all you want.

  • for disk IO, if you use phy: devices, you can use iostat to see the usage of each device. if you use file-based backends, it would be easier to check the userspace daemons. maybe iotop would help. in any case, if aggregate usage is all you need, just measure the disk usage seen at Dom0
  • for network, if you can measure at peth0, that would give you the aggregate usage. if you need stats for each DomU, check the respective tun devices.

Which parts of my network infrastructure should I run in dom0, and which should I run in domU?

Network infrastructure, such as DHCP, DNS and OpenVPN servers, should all run in domUs. Installing such services in dom0 is dangerous, as if the services are compromised, an attacker could access other running VMs.

As a general rule, a domU should be used to perform a small, independent task. Therefore if the OS crashes, or is compromised, the effect is constrained to a small part of your setup. This makes it easier to find the problem, and then fix.

When designing such setups, with bridge networking, I often find it easier to think of dom0 as a switch or router, and domU like any other physical server on your network.

In your setup you're making dom0 act as router/firewall. Your problem is that probably you haven't setup ip forwarding and NAT on dom0 to allow domU internet access.

Note that (if you want) you could also have dom0 act like a switch. In that scenario you'd need another domU, with two network interfaces connected to both dom0's eth0 and eth1, acting as router/firewall.

I have been trying to get a HVM DomU running and being able to connect to a vlan. I am starting to get the feeling that at least the hw emulations I have tried do not supoprt vlans. Also all the things I have found online would have me creating the vlans inside the Dom0 and pushing them to the DomU's as regular interface

You could always have a trunk port in your dom0 and create bridges for each VLAN for xen. You can even script it so you can add it to boot time. If you have the VLAN trunk set up, you can create bridges as follows.

For this example, my trunk interface is on eth0 and the vlan I am adding is 2.


# vconfig add eth0 2 # brctl addbr xenbr2 # brctl addif xenbr2 eth0.2 # ifconfig eth0.2 up # ifconfig xenbr2 up

Now all you would add in the domU configuration file is:

vif=['bridge=xenbr2']

And you would be on VLAN 2. Otherwise I'm pretty sure you would have to pass through the network card to get VLAN access. You can also script this and give it a space separated list of VLANs and loop it through. I will leave this up to you though.