XenDom0VLANstoDomUVirtualNICs: Difference between revisions

From Xen
Jump to navigationJump to search
mNo edit summary
mNo edit summary
 
Line 413: Line 413:
[[Category:Xen]]
[[Category:Xen]]
[[Category:Users]]
[[Category:Users]]
[[Category:Examples]]
[[Category:Example]]
[[Category:Beginners]]
[[Category:Beginners]]
[[Category:HowTo]]
[[Category:HowTo]]

Latest revision as of 11:11, 4 May 2012


Introduction

Icon todo.png To Do:

Describe XenDom0VLANstoDomUVirtualNICs here.


Examples

These are very simple instructions for mapping VLANs on the Dom0 through to the DomU NICs. These are Redhat/CentOS specific, and in this example the NICs were mapped through to a HVM Windoze box.

Other descriptions to help being picked up by search engines is:

  1. VLAN trunk to virtual machine mapping
  2. VLANs to DomU interfaces

Basic Flow:

  1. Create a bridge group for each .1q interface you want, and add the interfaces to the bridge.
  2. Map each virtual NIC to the bridge group.
  3. Don't forget to set the MTU in Windoze


  1. Configure the bond interface, and create VLAN interfaces on the bond interface.
  2. Create a bridge and map each vlan interface into the bridge.

All the files for this are at /etc/sysconfig/network-scripts/. (This examples creates VLAN 900-903.) You will need to change the HWADDR to match you NIC cards.

These are all the related files: 

/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth1
/etc/sysconfig/network-scripts/ifcfg-bond0
/etc/sysconfig/network-scripts/ifcfg-bond0.900
/etc/sysconfig/network-scripts/ifcfg-bond0.901
/etc/sysconfig/network-scripts/ifcfg-bond0.902
/etc/sysconfig/network-scripts/ifcfg-bond0.903
/etc/sysconfig/network-scripts/ifcfg-br900
/etc/sysconfig/network-scripts/ifcfg-br901
/etc/sysconfig/network-scripts/ifcfg-br902
/etc/sysconfig/network-scripts/ifcfg-br903

1.1 These files configure the real ethernet ports to be part of an active/standby 'bond' interface.

/etc/sysconfig/network-scripts/ifcfg-eth0 


# Intel Corporation 82541GI Gigabit Ethernet Controller
DEVICE=eth0
BOOTPROTO=none
HWADDR=00:15:17:28:f6:34
ONBOOT=yes
TYPE=Ethernet
#ETHTOOL_OPTS="autoneg off speed 100 duplex full"
MASTER=bond0
SLAVE=yes
USERCTL=no

/etc/sysconfig/network-scripts/ifcfg-eth1 


# Intel Corporation 82566DM-2 Gigabit Network Connection
DEVICE=eth1
BOOTPROTO=none
HWADDR=00:15:17:28:f6:36
ONBOOT=yes
TYPE=Ethernet
#ETHTOOL_OPTS="autoneg off speed 100 duplex full"
MASTER=bond0
SLAVE=yes
USERCTL=no

1.2 This is the bond interface configuration.

Here the bond0 interface, in Dom0, is actually getting a IP via dhcp. You may want to set BOOTPROTO=none if you don't want this. The 'virtual mac' address is set here. Make sure this is unique.

/etc/sysconfig/network-scripts/ifcfg-bond0 


DEVICE=bond0
BOOTPROTO=dhcp
ONBOOT=yes
USERCTL=no
#PEERDNS=no
IPV6INIT=no
MACADDR=52:54:00:12:45:56
BONDING_OPTS="mode=1 miimon=100 primary=eth0"

1.3 Now create the bridges.

/etc/sysconfig/network-scripts/ifcfg-br900 


DEVICE=br900
TYPE=Bridge
#STP=on
ONBOOT=yes
BOOTPROTO=none

... 


DEVICE=br903
TYPE=Bridge
#STP=on
ONBOOT=yes
BOOTPROTO=none

1.4 Now the creation of all the VLAN interfaces, which are put into the bridges.

/etc/sysconfig/network-scripts/ifcfg-bond0.900 


DEVICE=bond0.900
PHYSDEV=bond0
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
BRIDGE=br900

... 


DEVICE=bond0.903
PHYSDEV=bond0
ONBOOT=yes
BOOTPROTO=none
VLAN=yes
BRIDGE=br903

2. Load the bond-ing module.The main thing to add is below (rest of the bond is set in the file above). Reload or modprobe. 


alias bond0 bonding

3. Finally, the important line in the Xen xm config file is: 


vif = [ 'type=ioemu, bridge=br900', 'type=ioemu, bridge=br901',
'type=ioemu, bridge=br902', 'type=ioemu, bridge=br903' ]

4. Don't forget to set the MTU to 1494 using the 'regedit' if you are using windoze. 

If you are using Centos/Redhat you can also use 'virsh' which has a XML virtual machine definition.  'virsh' seems to be strongly supported by Redhat, and XML is fairly clean, so instead of just using the 'xm' style files do this:



<domain type='xen'>
  <name>vm_name</name>
  <memory>256000</memory>
  <bootloader>/usr/bin/pygrub</bootloader>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <vcpu>2</vcpu>
  <devices>
    <disk type='file' device='disk'>
      <driver name='tap'/>
      <source file='/home/xen/my_vm_centos52.img'/>
      <target dev='xvda' bus='xen'/>
    </disk>
    <interface type='bridge'>
      <source bridge='br900'/>
      <mac address='00:16:2e:62:f0:1c'/>
      <script path='vif-bridge'/>
    </interface>
    <interface type='bridge'>
      <source bridge='br901'/>
      <mac address='00:16:2e:62:f1:1c'/>
      <script path='vif-bridge'/>
    </interface>
    <interface type='bridge'>
      <source bridge='br902'/>
      <mac address='00:16:2e:62:31:1c'/>
      <script path='vif-bridge'/>
    </interface>
    <interface type='bridge'>
      <source bridge='br903'/>
      <mac address='00:16:2e:62:31:3c'/>
      <script path='vif-bridge'/>
    </interface>
  </devices>
</domain>

Do 'virsh create vm_name.xml', 'virsh list', and 'virsh autostart vm_name'

regards, Dave Seddon

Here are some additional notes on roughly the same implementation using QEMU/KVM. The key here is again, setting up the correct bridge groups and linking the virtual NICs to these.

These are some KVM/QEMU instructions for giving virtual machines multiple NICs bound to multiple physical NICs. Eg. 

Eth0 - Br0 - tap0 - Virtual NIC 0
Eth1 - Br1 - tap1 - Virtual NIC 1


Linux host configuration (debian/ubuntu) This setups up the two bridge interfaces (no STP) 

# The primary network interface
auto br0
iface br0 inet dhcp
        bridge_stp off
        bridge_ports eth0
iface eth0 inet static
address 0.0.0.0
netmask 0.0.0.0
auto eth0
iface eth1 inet static
        address 0.0.0.0
        netmask 0.0.0.0
auto eth1
auto br1
iface br1 inet static
        address 0.0.0.0
        netmask 0.0.0.0
        bridge_stp off
        bridge_ports eth1


To avoid KVM having to create the tap devices when it starts up (which required root), setup an init script to generate the interfaces with the correct ownership. You will need to change 'das' to whatever username you are using to run KVM. Recommend starting this script after the /etc/init.d/network runs. 

root@das-hp:/home/das/qemu# cat /etc/init.d/make_tap_devices.sh
#!/bin/sh
#make tap0
/usr/sbin/tunctl -u das
#make tap1
/usr/sbin/tunctl -u das


KVM normally calls this script when creating tap interfaces: (this script matches the bridge/switch on the default route - so usually br0, for example) 

root@das-hp:/etc/kvm# cat kvm-ifup
#!/bin/sh
switch=$(ip route ls | awk '/^default / { for(i=0;i<NF;i++) { if ($(i)
== "dev") print $(i+1) }}')
/sbin/ifconfig $1 0.0.0.0 up
/usr/sbin/brctl addif ${switch} $1
exit 0


However, we need a script that will match the first argument and change this to the correct bridge/switch: e.g. This script is called via kvm when creating a new tap interface kvm-ifup-tap tap0 -> add to bridge br0 kvm-ifup-tap tap1 -> add to bridge br1 

root@das-hp:/etc/kvm# cat kvm-ifup-tap
#!/bin/sh
#this script is called via kvm when creating a new tap interface
# $1 = tap0, or tap1, or whatever tap device
bridge=$(echo $1 | sed -e 's/tap//')
#bring the tap interface up
/sbin/ifconfig $1 0.0.0.0 up
#add the tap interface to the bridge
/usr/sbin/brctl addif br${bridge} $1
exit 0


Then finally the command to run all this is: 

sudo /usr/bin/kvm -m 1100 \
        -localtime \
        -usbdevice tablet \
        -net nic,vlan=0,model=e1000,macaddr=52:54:00:12:34:57 -net tap,vlan=0,ifname=tap0,script=/etc/kvm/kvm-ifup-tap \
        -net nic,vlan=1,model=e1000,macaddr=52:54:00:12:34:58 -net tap,vlan=1,ifname=tap1,script=/etc/kvm/kvm-ifup-tap \
        /home/das/qemu/winxp10g.img.qcow

Notes: 1. Make sure the 'vlan=X' the Xes match. This is how the virtual nic is associated with the tap interface. The NIC does not generate tagged frames. If the second NIC line above also had vlan=0, then all the NICs are effectively part of the same VLAN (and causes loops) 2. The above command is staring windoze, however it will work for other OSes too (e.g. Juniper olives) 3. The MAC addresses are hard coded to make sure DHCP address assignment is consistent) Alternatively, you can replace the existing kvm-ifup script (so you don't need to specify this to kvm): 

mv kvm-ifup not.kvm-ifup
ln -s kvm-ifup-tap kvm-ifup


{{{
root@das-hp:/etc/kvm# ls -la
total 28
drwxr-xr-x   3 root root  4096 2009-04-16 11:20 .
drwxr-xr-x 174 root root 12288 2009-04-16 10:44 ..
lrwxrwxrwx   1 root root    12 2009-04-16 11:20 kvm-ifup -> kvm-ifup-tap
-rwxr-xr-x   1 root root   294 2009-04-16 11:19 kvm-ifup-tap
-rwxr-xr-x   1 root root   181 2008-04-23 04:35 not.kvm-ifup
drwxr-xr-x   2 root root  4096 2008-10-06 13:13 utils


KVM command becomes: 

sudo /usr/bin/kvm -m 1100 \
        -localtime \
        -usbdevice tablet \
        -net nic,vlan=0,model=e1000,macaddr=52:54:00:12:34:57 -net tap,vlan=0,ifname=tap0 \
        -net nic,vlan=1,model=e1000,macaddr=52:54:00:12:34:58 -net tap,vlan=1,ifname=tap1 \
        /home/das/qemu/winxp10g.img.qcow

Notes: 1. Make sure the 'vlan=X' the Xes match. This is how the virtual nic is associated with the tap interface. The NIC does not generate tagged frames. If the second NIC line above also had vlan=0, then all the NICs are effectively part of the same VLAN (and causes loops) 2. The above command is staring windoze, however it will work for other OSes too (e.g. Juniper olives) 3. The MAC addresses are hard coded to make sure DHCP address assignment is consistent)

Some other interesting output: 

root@das-hp:/etc/kvm# ifconfig -a
br0       Link encap:Ethernet  HWaddr 00:1a:4b:5a:b4:96
          inet addr:XX.XX.XX.36  Bcast:XX.XX.XX.255  Mask:255.255.255.0
          inet6 addr: fe80::21a:4bff:fe5a:b496/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6892185 errors:0 dropped:0 overruns:0 frame:0
          TX packets:682 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:896362303 (854.8 MB)  TX bytes:274918 (268.4 KB)
br1       Link encap:Ethernet  HWaddr 00:ff:eb:19:39:9b
          inet6 addr: fe80::1211:ff:fe00:477/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:576 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:85748 (83.7 KB)  TX bytes:468 (468.0 B)
eth0      Link encap:Ethernet  HWaddr 00:1a:4b:5a:b4:96
          inet6 addr: fe80::21a:4bff:fe5a:b496/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:3014 errors:0 dropped:0 overruns:0 frame:0
          TX packets:936 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:422764 (412.8 KB)  TX bytes:219484 (214.3 KB)
          Interrupt:16
eth1      Link encap:Ethernet  HWaddr 10:11:00:00:04:77
          inet6 addr: fe80::1211:ff:fe00:477/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:291 errors:3464688 dropped:0 overruns:0 frame:0
          TX packets:291 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:43297 (42.2 KB)  TX bytes:47029 (45.9 KB)
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:16002 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16002 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:12053229 (11.4 MB)  TX bytes:12053229 (11.4 MB)
tap0      Link encap:Ethernet  HWaddr 00:ff:62:c1:3b:12
          inet6 addr: fe80::2ff:62ff:fec1:3b12/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:174 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:25728 (25.1 KB)  TX bytes:468 (468.0 B)
tap1      Link encap:Ethernet  HWaddr 00:ff:eb:19:39:9b
          inet6 addr: fe80::2ff:ebff:fe19:399b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:90 errors:0 dropped:0 overruns:0 frame:0
          TX packets:90 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:13110 (12.8 KB)  TX bytes:13086 (12.7 KB)



root@das-hp:/etc/kvm# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.001a4b5ab496       no              eth0
                                                        tap0
br1             8000.00ffeb19399b       no              eth1
                                                        tap1
Retrieved from "https://wiki.xenproject.org/index.php?title=XenDom0VLANstoDomUVirtualNICs&oldid=3474"