Xen FAQ Networking
|
Needs Review
This page is very out-of-date. Many of the issues described on this page have been resolved. |
Networking Issues
What is veth, vif or xenbr0?
You should read XenNetworking http://wiki.xen.org/wiki/XenNetworking
Why can't I ssh into or ping a newly created domain?
In the default configuration we rely on the Linux bridge-utils in domain 0 to set up virtual networking. After you've created a new domain (e.g., domain 1) you should be able to run ifconfig in domain 0 and see an interface with a name like vif1.0; you should also be able to check that bridging is working by typing brctl show xen-br0. Finally, you can check the IP confiuration in the new domain by logging into it via the console (xm console) and running standard tools such as ifconfig and route.
TCP and UDP checksum errors, ping but nothing else, ipsec tunnels don't form, DNAT translation doesn't work
Try running the following command in each domU:
ethtool -K eth0 tx off
This command disable TX checksumming
To check checksum problems, you can use tcpdump:
tcpdump -vv -n -i eth0
Read http://lists.xensource.com/archives/html/xen-users/2006-04/msg00032.html
This may or may not work.
Xen and Shorewall
There is a document about configuring Shorewall in Dom0 at http://www.shorewall.net/Xen.html
http://www1.shorewall.net/XenMyWay.html can be useful also.
Too many vethX and vif0.X
By default, 8 vethX and vif0.X are created. These interfaces are "cheap" but clutter list of interfaces. If you want to allocate only the necessary number, pass netloop.nloopbacks=NUMBER parameter to kernel command line.
I can't use more than 3 network interfaces in domU
This is a limitation in Xen 3. Xen 3.1 supports 8 network interfaces.
Bridging
Which Mechanism is used by Xen bridging to handle packets coming from various VMs to forward them to their destination
Nothing. Xen by itself does not handle bridge. dom0 OS does that. On Linux dom0 : http://www.linuxfoundation.org/en/Net:Bridge On opensolaris dom0: http://opensolaris.org/os/project/crossbow/
IP Determination
I want to know the IP of a running VM in XEN. Is there any way to have this without login to that VM?
Yes. First, you need to find the MAC address of the domU. This can be done by running:
xl network-list <domU name>
which should produce an output similar to:
Idx BE MAC Addr. handle state evt-ch tx-/rx-ring-ref BE-path 0 0 00:16:3E:F7:D6:E7 0 4 6 16238/16237 /local/domain/0/backend/vif/163/0
The domU's MAC address is 00:16:3E:F7:D6:E7
You now need to snoop the bridge for domU's MAC. For example:
# tcpdump -n -i eth0 ether src 00:16:3e:f7:d6:e7 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes 15:54:56.419482 IP 10.0.0.10 > 10.0.0.1: ICMP echo reply, id 5443, seq 1, length 64 15:54:57.422349 IP 10.0.0.10 > 10.0.0.1: ICMP echo reply, id 5443, seq 2, length 64
Then you know that domU has IP address 10.0.0.10.
NAT
I managed to configure NAT on dom0 but this does not work properly. Outgoing traffic from domU is seen with the original domU ip address instead of the dom0 ip address and the requests can't get back to the domU.
I figured out MASQUERADING was not set.
The following rule needs to be set:
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
SSL/VPN
Why can't I use openvpn with a xen guest? I can't load the tun module
From openvpn perspective, the requirements for openvpn on xen domU is the same as openvpn on native Linux. If you can't load the tun module, then you need to get a kernel that supports it.
The easiest way is to use a distro that supports it. For example, I'm using RHEL/Centos 5.3 domU, loaded with pygrub, and they can run openvpn just fine.
Another alternative is compile your own kernel with tun/tap support.
I want to ask how to mount one storage device to 2 guests? When I try to create vm handle everything is fine I can create one vdi and vbds for every guest. When I start first machine everything is ok, but when I try to start second one it says that
ERROR: 2 INTERNAL_ERROR Device 2051 (vbd) could not be connected. Device /dev/mapper/test_vg-test64_454 is mounted in a guest domain, and so cannot be mounted now.
Ideas how I can share one device to two or more virtual machines? I don't want network solution like nfs,iscsi ... etc but instead use ocfs2. How I can set mode to w! ?
First of all, you DO realize that sharing a block device without some kind of cluster file system could lead to data corruption? If you want to share it anyway, you can try changing the mode to "r"(for read only) or "w!" (to force read-write multiple mount).
in your domU.cfg:
'phy:/dev/data/bla-disk,sda1,w' => 'phy:/dev/data/bla-disk,sda1,r'
I'm looking for a way to monitor network activities of processes in Guest OS. I want to get a list of Guest OS processes that open TCP connections to other machines (like "lsof" command).
If you're thinking about doing on from dom0, that's not possible. You need something that runs on domU for that, possibly by using snmpd and extending it to run "netstat -anp --tcp". Other host (including dom0) can then collect the information using snmp.
Also, have a look at Versiera, it provides what you are looking for including, user IDs, inbound/outbound communications, IPv4, IPV6, etc. There are many more capabilities. Versiera is not open-source, but the Internet self-manage service is free.
I am attempting to gather stats on usage of the "metal", by which I mean the physical host's hardware. I would like to know the CPU, IO, and network stats for the hardware.
All DomU's IO passes through Dom0. there you can measure all you want.
- for disk IO, if you use phy: devices, you can use iostat to see the usage of each device. if you use file-based backends, it would be easier to check the userspace daemons. maybe iotop would help. in any case, if aggregate usage is all you need, just measure the disk usage seen at Dom0
- for network, if you can measure at peth0, that would give you the aggregate usage. if you need stats for each DomU, check the respective tun devices.
I have some trouble finding the best solution to my networking requirements. I want to have the following things:
- dom0 : have 2 physical networks devices * 1 eth with public IP (static) * 1 eth with private IP (static)
- domU * 1 eth with private IP
- I also want openVPN solution to let people outside the private network have access to it.
- A DHCP server is required so that domU get their IP from it.
I have debian lenny and xen 3.2 installed and working. Actually openvpn and dhcp are on the dom0. All is fine *except* that domU don't have access to internet (this is my main problem). My current config use network-bridge netdev=eth1 (eth1 have a static private ip).
It is perhaps better to have dhcp and openvpn server in a domU, feel free to suggest what you think is the best choice (and the config that go with it) :)
When designing such setups, with bridge networking, I often find it easier to think of dom0 as a switch or router, and domU like any other physical server on your network.
In your setup you're making dom0 act as router/firewall. Your problem is that probably you haven't setup ip forwarding and NAT on dom0 to allow domU internet access.
Note that (if you want) you could also have dom0 act like a switch. In that scenario you'd need another domU, with two network interfaces connected to both dom0's eth0 and eth1, acting as router/firewall.
== I have been trying to get a HVM DomU running and being able to connect to a vlan. I am starting to get the feeling that at least the hw emulations I have tried do not supoprt vlans. Also all the things I have found online would have me creating the vlans inside the Dom0 and pushing them to the DomU's as regular interface ==
You could always have a trunk port in your dom0 and create bridges for each VLAN for xen. You can even script it so you can add it to boot time. If you have the VLAN trunk set up, you can create bridges as follows.
For this example, my trunk interface is on eth0 and the vlan I am adding is 2.
# vconfig add eth0 2 # brctl addbr xenbr2 # brctl addif xenbr2 eth0.2 # ifconfig eth0.2 up # ifconfig xenbr2 up
Now all you would add in the domU configuration file is:
vif=['bridge=xenbr2']
And you would be on VLAN 2. Otherwise I'm pretty sure you would have to pass through the network card to get VLAN access. You can also script this and give it a space separated list of VLANs and loop it through. I will leave this up to you though.
