XCP Ubuntu PCIPassthrough
The below is only applicable to PV hosts. In this example the OS used was Ubuntu 10.04 LTS.
I don't believe you can do USB-only pass through with XCP 1.0. However, you can do PCI pass-through and pass-through the entire USB controller device.
Read this guide: it contains most of the answers, even if it's hard to parse: Xen_PCI_Passthrough
NOTE: Although this entire procedure worked, in the end I have to admit I found that PCI passthrough was too complex and not reliable enough for my purposes, so please give it a try, but I recommend doing it on a test machine.
Confirm that your dom0 supports pciback
I used XCP 1.0 in my case, which does support pciback, so you can skip this if you're on XCP.
# (on dom0) # Look for 'pciback' here ls /sys/bus/pci/drivers/
Confirm domU supports pciback
Now check your Linux domU (guest OS) for pci-passthrough support (needs xen-pcifront module). Your Ubuntu system must have xen-pcifront/xen_pcifront installed in the kernel for pci pass-through to work. Basically your kernel version must be v2.6.37+ to have it included by default.
# (on domU) # check the version uname -a
"uname -a" should return a kernel version greater than 2.6.37.
Custom domU pciback kernel
If your kernel version is less than 2.6.37, and there are no readily available replacement kernels, you can compile a custom kernel yourself.
Compiling the kernel
No Ubuntu 10.04 packages exist for a kernel of this version. I also tried the kernels located here: http://kernel.ubuntu.com/~kernel-ppa/mainline/, but apparently they're not compiled with Xen support or something because my XCP refused to boot the kernel complaining about missing modules... so I finally gave up and compiled my own kernel:
# (on domU) # v2.6.37+ linux has pcifront built in: # if you have a lower version, you need a newer kernel # here's compile instructions I used on Lucid 10.04: # reference: http://linuxtweaking.blogspot.com/2011/02/how-to-compile-kernel-from-kernelorg-in.html # # If you feel like living dangerously/lazily, you can try to use the 32-bit kernel I compiled.. # I've put up a link here, but naturally this comes with NO warranty at all, and Ubuntu specifically states # compiling your own kernel voids any potential warranty/support.. so there's that.. # anyways: # http://dev.locatrix.com/linux-headers-2.6.37_2.6.37-10.00.Custom_i386.deb # http://dev.locatrix.com/linux-image-2.6.37_2.6.37-10.00.Custom_i386.deb # so skip most of the below and follow along from where it starts with dpkg -i for the above files cd ~ # or somewhere apt-get install fakeroot kernel-wedge build-essential makedumpfile kernel-package libncurses5 libncurses5-dev apt-get build-dep --no-install-recommends linux-image-$(uname -r) mkdir src cd src wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.37.tar.gz tar xvf linux-2.6.37.tar.gz cd linux-2.6.37 # basically I left all the defaults alone.. seemed to already include xen and everything else #for some god unknown reason I did have to uncheck the option labeled "one laptop per child" (OLC) support... no clue.. make menuconfig export CONCURRENCY_LEVEL=3 make-kpkg clean time fakeroot make-kpkg --initrd kernel-image kernel-headers cd .. ls -l # note the package file names and install them dpkg -i linux-headers-2.6.37_2.6.37-10.00.Custom_i386.deb dpkg -i linux-image-2.6.37_2.6.37-10.00.Custom_i386.deb update-initramfs -c -k 2.6.37 update-grub # note down the new kernel/init versions ls -la / # e.g. /vmlinuz-2.6.37 /initrd.img-2.6.37 #shutdown halt
Configure XCP to use the new kernel
Here's how I re-configured my dom0 XCP to use the new kernel (this is the same as you configure it in the first place for which kernel to use):
# BACK on dom0 #find VM's uuid xe vm-list #take down a copy of this stuff before you change it xe vm-param-list uuid=MYUUID #specifically we'll modify the bootloader xe vm-param-list uuid=MYUUID | grep PV-bootloader-args #make sure your VM is now set to boot from the new kernel #this may or may not apply to you.. but I configured my VM to have the kernel/ramdisk set this way. #make sure to modify the below to match your exact kernel/ramdisk and remember the above in case you need to change it back. xe vm-param-set uuid=MYUUID PV-bootloader-args="--kernel=/vmlinuz-2.6.37 --ramdisk=/initrd.img-2.6.37" xe vm-param-list uuid=MYUUID | grep PV-args # note the args.. modify to add "iommu=soft swiotlb=force", for example: xe vm-param-set uuid=MYUUID PV-args="root=/dev/MYBOOTDRIVE ro quiet iommu=soft swiotlb=force" Now search for the PCI device you want to passthrough. I wanted to passthrough a USB device, so that's what I'm doing here: # (on dom0) #search for the device/pci information.. couple different ways: #easy way to identify pci devices: lspci -tv #best way to get the exact pci number, the device number should be from here: ls /sys/bus/pci/devices #some more general info for USB for confirmation #for usb 2.0 dmesg | grep ehci_hcd #e.g. ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 21 (level, low) -> IRQ 21 #for usb 1.1 dmesg | grep uhci_hcd #e.g. uhci_hcd 0000:00:1d.3: irq 23, io base 0x0000ff20 #note down the pci address 0000:00:1d.7 # how I found the usb 1.1 UPS device: dmesg | tail #usb 2-1: USB disconnect, address 2 #usb 2-1: new low speed USB device using uhci_hcd and address 3 # regardless really use the device number from /sys/bus/pci/devices above
Setup PCI passthrough
Now set the PCI devices to pass through. The below seems to be correct. The 0/ seems to be the virtual bus number.. so from what I understand it should always be 0.
# (on dom0) #the below is a comma separated list of each PCI device to pass-through.. I passed through all of my USB controllers that I found #the list could be just one device, like "other-config:pci=0/0000:00:1d.0".. I passed through all of them: xe vm-param-set other-config:pci=0/0000:00:1d.0,0/0000:00:1d.1,0/0000:00:1d.2,0/0000:00:1d.3,0/0000:00:1d.7 uuid=MYUUID #You can later remove this setting if needed xe vm-param-remove param-name=other-config param-key=pci uuid=MYUUID
Start up the VM again
# (on dom0) xe vm-start uuid=MYUUID
Watch the xcp system logs as the machine boots to catch the pciback passthrough messages
# (on dom0) tail -f /var/log/kern.log I received messages like "FLR functionality not supported", but apparently that shouldn't stop it from working. The bottom of this page talks about FLR: XenPCIpassthrough The below allows live attaching a pci device to the VM. This also worked once I fixed up the linux kernel as above. <pre> # (on dom0) xl list # note down the "Name" for your VM.. use below... # again, the PCI device number here is from "ls /sys/bus/pci/devices" (with "lspci -tv" helping you find the right one) xl pci-attach mymachinename 0000:00:1d.7 # there's other handy xl commands.. # xl pci-detach mymachinename 0000:00:1d.7 # xl pci-list mymachinename