https://wiki.xenproject.org/api.php?action=feedcontributions&user=Powerhouse&feedformat=atomXen - User contributions [en]2024-03-28T09:10:57ZUser contributionsMediaWiki 1.31.3https://wiki.xenproject.org/index.php?title=Category:HowTo&diff=7829Category:HowTo2013-05-16T07:10:45Z<p>Powerhouse: /* Windows 7/XP/Vista/Server 2008 */</p>
<hr />
<div>[[File:XenFuPandaWikiHowTo.png]]<br />
<br />
'''Go [[#pages|here]] to see a list of documents in this category.'''<br />
----<br />
This category contains HowTo documents. HowTo documents are detailed "how to" documents on specific subjects. <br />
__NOTOC__<br />
== External HowTo's ==<br />
<br />
{{Anchor|XCP}}<br />
== XCP ==<br />
* [http://grantmcwilliams.com/tech/virtualization/xcp-howtos Grant McWilliam's XCP HowTos]<br />
Other XCP HowTo's:<br />
* [[XCP HowTo, Index of]]<br />
<br />
== File System ==<br />
* [http://www.virtuatopia.com/index.php/Creating_and_Booting_a_Xen_Guest_domainU_using_an_NFS_Mounted_Root_Filesystem How to Create and Boot a Xen Guest domainU using an NFS Mounted Root Filesystem]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Xen_Virtual_Guest_Filesystem_on_a_Disk_Image_(Cloning_Host_System) How to Build a Xen Virtual Guest Filesystem on a Disk Image (Cloning Host System)]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Xen_Virtual_Guest_Filesystem_using_Logical_Volume_Management_(LVM) How to Build a Xen Virtual Guest Filesystem using Logical Volume Management (LVM)]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Xen_Virtual_Guest_Filesystem_on_a_Physical_Disk_Partition_(Cloning_Host_System) How to Build a Xen Virtual Guest Filesystem on a Physical Disk Partition (Cloning Host System)]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Debian_or_Ubuntu_Xen_Guest_Root_Filesystem_using_debootstrap How to Build a Debian or Ubuntu Xen Guest Root Filesystem using debootstrap]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Xen_Guest_Root_Filesystem_using_yum_and_rpm Building a Xen Guest Root Filesystem using yum and rpm]<br />
<br />
== DomainU Guest ==<br />
* [http://grantmcwilliams.com/tech/virtualization/xen-howtos/ Grant McWilliam's Xen HowTos]<br />
* [http://www.virtuatopia.com/index.php/Using_QEMU_Disk_Images_for_Xen_DomainU_Systems How to Use QEMU Disk Images for Xen DomainU Systems]<br />
* [http://www.virtuatopia.com/index.php/Building_a_Xen_Guest_Domain_using_Xen-Tools How to Build a Xen Guest Domain using Xen-Tools]<br />
* [[Setting boot order for domUs]]<br />
<br />
== Windows 7/XP/Vista/Server 2008 ==<br />
* [http://www.virtuatopia.com/index.php/Installing_and_Running_Windows_7_as_a_Xen_HVM_domainU_Guest Installing and Running Windows 7 as a Xen HVM domainU Guest]<br />
* [http://www.virtuatopia.com/index.php/Installing_and_Running_Windows_XP_or_Vista_as_a_Xen_HVM_domainU_Guest Installing and Running Windows XP or Vista as a Xen HVM domainU Guest]<br />
* [http://www.virtuatopia.com/index.php/Virtualizing_Windows_Server_2008_with_Xen Virtualizing Windows Server 2008 with Xen]<br />
* [http://www.valent-blog.eu/2008/01/13/full-virtualization-di-windows-con-xen/ Create a windows 32-bit domU (in italian)]<br />
* [http://forums.linuxmint.com/viewtopic.php?f=42&t=112013 Installing Windows 7 as a Xen HVM guest with VGA passthrough on Linux Mint 13/14]<br />
<br />
== Consoles ==<br />
* [http://www.virtuatopia.com/index.php/Configuring_a_VNC_based_Graphical_Console_for_a_Xen_Paravirtualized_domainU_Guest How to Configure a VNC based Graphical Console for a Xen Paravirtualized domainU Guest]<br />
* [http://www.virtuatopia.com/index.php/Running_and_Connecting_to_VNC_Servers_on_a_Xen_Guest_(domainU)_System How to Run and Connect to VNC Servers on a Xen Guest (domainU) System]<br />
<br />
== DomainU Migration ==<br />
* [http://www.virtuatopia.com/index.php/Migrating_Xen_domainU_Guests_Between_Host_Systems How to Perform Live Xen domainU Guest Migrations]<br />
* [[Install Xen 4.1.4 with Remus and DRBD on Ubuntu_12.10]]<br />
<br />
== USB Devices ==<br />
* [http://www.virtuatopia.com/index.php/Adding_USB_Devices_to_a_Xen_HVM_domainU_Guest Adding USB Devices to a Xen HVM Guest]<br />
<br />
== Install Xen on Various Linux Distros and [[OpenSolaris]] ==<br />
* [https://help.ubuntu.com/community/Xen Installing Xen on Ubuntu]<br />
* [http://wiki.xen.org/wiki/Debian_Guest_Installation_Using_Debian_Installer Installing Xen on Debian]<br />
* [http://rackerhacker.com/2011/08/05/xen-4-1-on-fedora-15-with-linux-3-0/ Installing Xen 4.1 on Fedora 15]<br />
* [http://www.suse.com/documentation/sled11/singlehtml/book_xen/book_xen.html Xen on openSUSE]<br />
* [http://www.netbsd.org/Ports/xen/howto.html NetBSD/xen Howto ]<br />
<br />
== Writing Good HowTos ==<br />
Good HowTo documents include<br />
* An '''overview''' that helps potential readers to determine quickly if a particular How-To matches their interests or needs. <br />
* Describe your '''intended audience''':<br />
* State the '''purpose''' of your HowTo: this will explain how the reader will benefit by reading it.<br />
* '''Preconditions''': inform your reader about any required knowledge, configuration, or resources they may need before stepping through your How-To.<br />
* '''Step-by-step instractions''': in a precise, step-by-step approach, walk your reader through the process. Make sure your reader can reproduce your intended result by following your exact steps. Make the learning process efficient by supplying sample code snippets or configuration details as necessary.<br />
<br />
==Add a page to this category==<br />
To add a page or uploaded file to a category, simply edit the page and add the following text at the bottom of the page (note that a page can be in several categories).<br />
<nowiki>[[</nowiki>{{ns:category}}:'''''HowTo''''']]<br />
<br />
{{Anchor|pages}}<br />
{{Languages|:Category:HowTo}}</div>Powerhousehttps://wiki.xenproject.org/index.php?title=Dom0_Kernels_for_Xen&diff=7828Dom0 Kernels for Xen2013-05-16T06:55:52Z<p>Powerhouse: /* Linux Distributions which have a Xen Dom0 kernel */</p>
<hr />
<div>If you don't know what "dom0" or a "dom0 kernel" is, take look at the detailed explanation [[Dom0|here]].<br />
<br />
This page contains information on Xen dom0 support, if you are looking for information on guest (domU) support see [[XenDomUSupport]].<br />
<br />
= Choosing a dom0 kernel =<br />
If your Linux distribution includes Xen and dom0 kernel it is recommended to use them. This provides benefits such as ease of install, good integration with the distribution, support from the distribution, and provision of security updates.<br />
<br />
Xen PV domU kernel versions can be different from dom0 kernel versions. There's no need to upgrade domU kernel when you upgrade Xen hypervisor and/or dom0 kernel. Xen hypervisor is backwards compatible, ie. it supports older domU kernels. If your domU distribution includes a Xen domU kernel, it's usually best to use it and not compile your own domU kernel.<br />
<br />
Also please notice that mainline vanilla Linux kernel already supports Xen. Instructions how to configure and build a Linux kernel for Dom0 and DomU support are reported here: [[ Mainline_Linux_Kernel_Configs ]] .<br />
<br />
= Linux Distributions which have a Xen Dom0 kernel =<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
|<tablestyle="width: 90%">|'''Distributions'''<br />
| '''Versions'''<br />
|-<br />
| [http://alpinelinux.org/ Alpine Linux]<br />
| [http://alpinelinux.org/node/13811 2.4.x]''[2]''<br />
|-<br />
| [http://www.debian.org/ Debian]<br />
| [http://www.debian.org/releases/etch/ 4.0 (Etch)], [http://www.debian.org/releases/lenny/ 5.0 (Lenny)], [http://www.debian.org/releases/squeeze/ 6.0 (Squeeze)]''[1]''<br />
|-<br />
| [http://fedoraproject.org/ Fedora]<br />
| [http://docs.fedoraproject.org/en-US/Fedora/16/html/Release_Notes/index.html 15 (Lovelock)]''[3]'', [http://docs.fedoraproject.org/en-US/Fedora/16/html/Release_Notes/index.html 16 (Verne)], [http://docs.fedoraproject.org/en-US/Fedora/17/html/Release_Notes/index.html 17 (Beefy Miracle)]''[1]''<br />
|-<br />
| [http://www.linuxmint.com/ Linux Mint]<br />
| 13, 14 ''[4]''<br />
|-<br />
| [http://www.opensuse.org/ OpenSUSE]<br />
| 10.x, 11.x, 12.x<br />
|-<br />
| [http://www.oracle.com/us/technologies/virtualization/oraclevm/index.html Oracle VM for x86 (OVS)]<br />
| All versions<br />
|-<br />
| [http://www.redhat.com Redhat Enterprise Linux (RHEL)]<br />
| 5.x<br />
|-<br />
| [http://www.centos.org CentOS and other RHEL clones]<br />
| 5.x<br />
|-<br />
| [http://www.novell.com/products/server/ SUSE Linux Enterprise Server (SLES)]<br />
| 10.x, 11.x<br />
|-<br />
| [http://www.ubuntu.com/ Ubuntu]<br />
| 11.10''[1]'', 12.04 LTS''[1]''<br />
|-<br />
| [http://www.xen.org/products/cloudxen.html Xen Cloud Project (XCP)]<br />
| All versions<br />
|-<br />
| [http://www.citrix.com/xenserver/ XenServer]<br />
| All versions<br />
|}<br />
<br />
[[Category:Alpine Linux]]<br />
[[Category:Debian]]<br />
[[Category:Ubuntu]]<br />
[[Category:Linux Mint]]<br />
[[Category:Fedora]]<br />
[[Category:Oracle Linux]]<br />
[[Category:CentOS]]<br />
[[Category:RHEL]]<br />
[[Category:OpenSuSE]]<br />
<br />
''[1]'' Please note that this kernel requires Xen >= 4.0.<br />
<br>''[2]'' See [http://wiki.alpinelinux.org/wiki/Xen_Dom0 wiki for more info]<br />
<br>''[3]'' Dom0 kernel provided via [http://fedorapeople.org/~myoung/dom0/ this] (unofficial) repository<br />
<br>''[4]'' Based on Ubuntu 12.04 and 12.10 resp.<br />
<br />
= Xen Dom0 support in BSDs and other Unix systems =<br />
<br />
This list is currently incomplete. We are in the process of verifying support for other Unixes. <br />
<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
|<tablestyle="width: 90%">|'''Distributions'''<br />
| '''Versions'''<br />
|-<br />
| NetBSD <br />
| 5.1, 6 [1], [2] <br />
|-<br />
| OpenSolaris<br />
| 2009.06 [3]<br />
|-<br />
| Illumos<br />
| Incomplete [4]<br />
|}<br />
<br />
[[Category:NetBSD]]<br />
[[Category:Solaris]]<br />
<br />
''[1]'' See [http://www.netbsd.org/ports/xen/howto.html#netbsd-dom0 NetBSD Xen HowTo] for more information<br />
<br>''[2]'' See [http://multiplicity.bsd.lv/netbsd-xen.html NetBSD Xen User Guide]<br />
<br>''[3]'' OpenSolaris has been discontinued by Oracle, but Xen dom0 support is present in the latest 2009.06 release and later development snapshots before OpenSolaris was discontinued.<br />
<br>''[4]'' Illumos [http://www.illumos.org] is a fork of OpenSolaris, and Xen dom0 support is present in the Illumos sources, but it's currently unmaintained and untested. Interested developers should be able to make it work again, because it was working earlier in OpenSolaris.<br />
<br />
= Mini-OS=<br />
Mini-OS is a tiny OS kernel distributed with the Xen hypervisor sources (see extras/mini-os in the Xen source code). It is mainly used as operating system for [[Device_Model_Stub_Domains|stub domains]] that are used for [[Dom0 Disaggregation]].<br />
<br />
= FAQ =<br />
Please see [[XenKernelFeatures]] for more information about available features in different Xen enabled kernels.<br />
<br />
Q: I've seen mentions of 'pv_ops' and 'xenlinux' kernels. Which one should I choose?<br />
<br />
Simple answer: pv_ops. Xen 4.0 switched to using Linux pv_ops based dom0 kernel as a default, and it's the most up-to-date and is undergoing active development.<br />
<br />
Longer answer: There are two different types of Xen dom0 capable kernels available today:<br />
<br />
* pvops kernels, featuring new rewritten Xen support based on the upstream (kernel.org) Linux pvops framework. This work has been included in upstream kernel.org kernel since Linux 2.6.37.<br />
* xenlinux kernels based on the "old" patches originally for Linux 2.6.18. These Xenlinux patches won't be integrated to upstream Linux.<br />
<br />
However, the pv_ops kernel requires Xen >=4.0. If you desperately need the Xenlinux patches, SLES/OpenSUSE has been forward-porting the patches.<br />
<br />
[[Category:Beginners|D]]<br />
[[Category:Users|D]]<br />
[[Category:Compatibility|D]]<br />
[[Category:Xen|D]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=VTd_HowTo&diff=7398VTd HowTo2013-04-02T15:04:54Z<p>Powerhouse: /* VT-d Enabled Systems */</p>
<hr />
<div><!-- MoinMoin name: VTdHowTo --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: VTdHowTo --><br />
<!-- Page revision: 00000104 --><br />
<!-- Original date: Sat Jul 30 03:13:48 2011 (1311995628000000) --><br />
<br />
{{TODO|This page is outdated. In particular the hardware sections need to be updated.}}<br />
<br />
__NOTOC__<br />
<!-- ## Please edit system and help pages ONLY in the moinmaster wiki! For more --><br />
<!-- ## information, please see [[MoinMaster]]:[[MoinPagesEditorGroup]]. --><br />
<!-- ##acl [[MoinPagesEditorGroup]]:read,write,delete,revert All:read --><br />
<!-- ##master-page:[[HelpTemplate]] --><br />
<!-- ##master-date:Unknown-Date --><br />
<!-- #format wiki --><br />
<!-- #language en --><br />
Please also check [[XenPCIpassthrough]] wiki page for more general information about Xen PCI passthru usage!<br />
<br />
VT-d Pass-Through is a technique to give a domU exclusive access to a PCI function using the IOMMU provided by VT-d. It is primarily targeted at HVM (fully virtualised) guests because PV (paravirtualized) pass-through does not require VT-d (altough it may be utilized too)<br />
<br />
== Xen 4.1 xl tools notes ==<br />
* Only devices with FLR capabilities are supported.<br />
* Passing through a PCI card without FLR capability will result an error.<br />
<br />
To check if your PCI devices have FLR function, check in this wiki, [[http://wiki.xen.org/wiki/Xen_PCI_Passthrough#How_can_I_check_if_PCI_device_supports_FLR_.28Function_Level_Reset.29_.3F How can I check if PCI device supports FLR (Function Level Reset) ?]]<br />
<br />
If you see output with "FLReset-" then your PCI device don't support FLR function. If output have "FLReset+" then it does.<br />
<br />
As this time of writing, June 2012, there are very very few PCI devices support FLR function.<br />
<br />
== Supported OSes ==<br />
* Dom0 (Host) OS: PAE, 64-bit<br />
* DomU (Guest) OS: 32-bit, PAE, 64-bit<br />
=== Tested DomO OSes ===<br />
* 64-bit host: 32/PAE/64 Linux/XP/Win2003/Vista guests<br />
* PAE host: 32/PAE Linux/XP/Win2003/Vista guests<br />
== VT-d Enabled Systems ==<br />
Note that in addition to the motherboard chipset and BIOS also your CPU must have support for IOMMU IO virtualization (VT-d)! Make sure to check your CPU vendors spec sheets for more information.<br />
<br />
CPUs known to work (with a motherboard from below list and a proper BIOS):<br />
<br />
(Intel reference the list of [http://ark.intel.com/search/advanced/?s=t&VTD=true compatible processors vt-d])<br />
<br />
* Intel Core2Duo (with VT-x)<br />
* Intel Core2Quad<br />
* Intel Core i7 (see link to Intel reference list above - most "K" versions don't support VT-d)<br />
* Intel Core i5 (vPro Brand)<br />
* AMD FX-8120 / FX-8150<br />
For VT-d enabling work on Xen, we have been using development systems using following Intel motherboards:<br />
<br />
* DQ35MP<br />
* DQ35JO<br />
* Notes on VT-d compatibility:<br />
** VT-d is enabled on the following chipsets:<br />
*** Intel Q35 (desktop / workstation)<br />
*** Intel Q45 (desktop / workstation)<br />
*** Intel X58 (desktop / workstation)<br />
*** Intel X79<br />
*** Intel 55x0 (server)<br />
*** Intel 3450 (workstation / server)<br />
* The following chipsets have VT-d capability in theory, but most OEMs (such as Asus and Gigabyte) do not have it enabled on boards based on these:<br />
** Intel X38 (desktop / workstation)<br />
** Intel X48 (desktop / workstation)<br />
** Intel 32x0 (server)<br />
* For Intel Desktop Boards, these have VT-d support enabled:<br />
** Intel DQ35JO<br />
** Intel DQ35MP<br />
** Intel DX38BT, DX48BT2 (BIOS 1554 or 1782 required, VT-d is known to not work in other versions; iommu=workaround_bios_bug required) <br />
** Intel DQ45CB (BIOS 0061 required, previous versions are known to cause problems)<br />
** Intel DQ45EK<br />
** Intel DX58SO<br />
** Intel DQ67SW<br />
* For ASUS Desktop Boards, these have VT-d support enabled, but Asus does NOT support Linux, so you are on your own with any Linux or Xen issues like broken BIOSes:<br />
** ASUS P5E-VM DO (Intel Q35 chipset) requires IGD to be enabled (otherwise DMAR-table becomes corrupted)<br />
** ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS update to correct DMAR-table issue<br />
** ASUS P6T documentation claims support, but does not work due to DMAR-table issue<br />
** ASUS P6T6 WS Revolution (Intel X58 chipset) incorrect bios DMAR-table<br />
** ASUS Sabertooth X58 (Intel X58 chipset) incorrect bios DMAR-table (2nd RMRR structure is incorrect)<br />
** ASUS Sabertooth X79 (with BIOS release 1203 and Marvell SATA controller disabled)<br />
** ASUS Z8NA-D6 (dual processor nehalem board) works<br />
* Most server boards based on the Tylersburg chipset (55x0) and few boards based on 32x0 should have working VT-d, known examples are:<br />
** Intel server board: [http://www.intel.com/support/motherboards/server/S3200SH/sb/CS-028345.htm S3210SHLX] (BIOS >R0044 required.)<br />
** Supermicro server mainboard: [http://www.supermicro.com/products/motherboard/QPI/5500/X8DT3-F.cfm X8DT3-F]<br />
** Supermicro server 5026T-TB: [http://www.supermicro.com/products/system/2U/5026/SYS-5026T-T.cfm] Only work USB 1.1.<br />
* These motherboards are known to have broken BIOS preventing IO virtualization (VT-d IOMMU) from working:<br />
** Supermicro X7SB4 (with official BIOS 1.2a) has broken ACPI DMAR table with zero length entries. BIOS version 1.3 Beta fixes the problem.<br />
** Samsung X460 laptop: BIOS doesn't provide DMAR table so VT-d cannot be used.<br />
* As far as we know, following OEM systems also have VT-d enabled. Feel free to add others as they become available.<br />
** Dell: [http://www.dell.com/content/products/category.aspx/optix?c=us&cs=555&l=en&s=biz Optiplex 755], [http://www.dell.com/us/business/p/optiplex-980/pd Optiplex 980]<br />
** HP Compaq: [http://h10010.www1.hp.com/wwpc/us/en/en/WF04a/12454-12454-64287-321860-3328898.html DC7800]<br />
** Fujitsu-Siemens: [http://facts.fujitsu-siemens.com/index.cfm?fuseaction=dspcontent&lid=134&page=3 Esprimo 5925]<br />
** Lenovo: [http://shop.lenovo.com/us/workstations/thinkstation/workstation/e20 ThinkStation E20] and Thinkserver TS200v<br />
** HP: [http://www.hp.com/united-states/campaigns/workstations/z420_features.html#.UG6055gxrug Z420 Workstation] <br />
*VT-d Compatible ASRock Motherboards (Socket 1155 and 2011) probably all Z68 and Z77 board's<br />
**ASRock H77 Pro4-M<br />
**ASRock Z68 Extreme4 Gen3<br />
**ASRock Z68 Professional Gen3<br />
**ASRock Z77 Pro3<br />
**ASRock Z77 Pro4<br />
**ASRock Z77 Extreme4<br />
**ASRock Z77 Extreme6<br />
**ASRock Z77 Professional<br />
**ASRock X79 Extreme9 (socket 2011)<br />
<br />
For Z77 Extreme4/Extreme6 users: ASRock introduced change in how VT-d setting can be modified within BIOS settings, this affects BIOS version >2.50 (for Extreme4 see [http://www.asrock.com/mb/Intel/Z77%20Extreme4/index.us.asp?cat=Download&os=BIOS]) and >2.30 (for Extreme 6 see [http://www.asrock.com/mb/Intel/Z77%20Extreme6/index.us.asp?cat=Download&os=BIOS]). So these BIOS versions '''do not disable VT-d support''', but just disable user ability to modify this setting within BIOS e.g. for non-K processors VT-d will be always enabled and for K-series disabled.<br />
<br />
* Gigabyte Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** GA-X79-UD5 (rev. 1.0)<br />
** GA-X79-UD7 (rev. 1.0)<br />
** GA-Z77X-D3H (rev. 1.0)<br />
* MSI 1155 Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** Z68A-GD80-G3<br />
** Z77A-GD80<br />
** Z77A-GD65<br />
** Z77A-GD55<br />
** Z77A-G45<br />
** Z77A-G43<br />
<br />
== AMD desktop chipsets with IOMMU support ==<br />
* AMD 890FX chipset supports IOMMU. Other 890 chipsets don't have IOMMU support!<br />
* AMD 990FX, 990X and 970 chipsets support IOMMU.<br />
* Even when the chipset supports IOMMU, the bios must have a ACPI IVRS table to enable the use of it! So actual support depends on the motherboard manufacturer. At the time of writing all motherboards seem to have a (beta)bios available supporting the IOMMU. A thread with user expiriences can also be found at this forum: http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/<br />
* Motherboards with a BIOS supporting the IOMMU(as reported by users):<br />
** ASUS Crosshair IV (reported working by Jens Krehbiel-Gräther)<br />
** ASUS Crosshair V Formula (reported working by Pavel Matěja)<br />
** ASUS F2A85-V PRO<br />
** ASUS M4A89TD Pro/USB3 (reported working by Jens Krehbiel-Gräther)<br />
** Asrock 890FX Deluxe3 (reported working by Jens Krehbiel-Gräther)<br />
** Biostar TA890FXE (from bios version 89FAD629.BST reported working by Joop Boonen, Konrad Rzeszutek Wilk)<br />
** Gigabyte GA-970A-UD3 (Bios F7)<br />
* Motherboards with a beta-bios available from tech-support that supports the IOMMU:<br />
** Gigabyte GA-890FXA-UD5<br />
** Gigabyte GA-890FXA-UD7<br />
** MSI 890FXA-GD70 (from beta-bios 1.75 reported working by Sander Eikelenboom)<br />
<br />
== AMD server (opteron) chipsets with IOMMU support ==<br />
* AMD SR5690 / SR5670 (Tyan S8212)<br />
== Caveat on Conventional PCI Device Pass-Through ==<br />
* The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.<br />
* PCIe devices do not have this restriction.<br />
== Notes on Compiling Source Code ==<br />
It is not the intention of this document to describe in detail how to compile and install Xen from source. Rather, the intention of this section is to highlight points of particular note when compiling from source for use with VT-d.<br />
<br />
=== Dom0 Kernel ===<br />
A prerequisite for binding devices to pciback, which is discussed in more detail below, is that pciback is compiled statically into or as a module for the dom0 kernel. This means that in the kernel config CONFIG_XEN_PCIDEV_BACKEND=y should be set to statically compile pciback into the kernel or CONFIG_XEN_PCIDEV_BACKEND=m should be set to compile pciback as a module.<br />
<br />
For pv-ops dom0, you can use pci-stub to hide device. Set CONFIG_PCI_STUB=y to build it into kernel or CONFIG_PCI_STUB=m to build as a module.<br />
<br />
=== Device Model ===<br />
When building the device model, also known as qemu-dm and qemu-xen, often automatically downloaded during compilation and found in the tools/ioemu-dir/ subdirectory of the xen tree, it is important that the development headers and libraries for libpci are installed. Otherwise qemu-dm will be built without pass-through support.<br />
<br />
If the device model has been compiled without pass-through support, the following error will show up in xend.log at run-time when pass-through is attempted.<br />
<br />
* <br />
<pre><br />
ERROR (XendDomainInfo:581) Device model didn't tell the vslots for PCI device<br />
</pre><br />
<br />
== VT-d boot parameter: iommu ==<br />
VT-d is disabled by default, to enable it, need 'iommu' parameter to enable it.<br />
<br />
* off|no|false|disable: Disable IOMMU (default)<br />
* pv: Enable IOMMU for PV domains<br />
* no-pv: Disable IOMMU for PV domains (default)<br />
* force|required: Don't boot unless IOMMU is enabled<br />
* workaround_bios_bug: Workaround some bios issues to still enable VT-d, don't guarantee security<br />
* pass-through: Enable VT-d DMA pass-through (no DMA translation for Dom0)<br />
* no-snoop: Disable VT-d Snoop Control<br />
* no-qinval: Disable VT-d Queued Invalidation<br />
* no-intremap: Disable VT-d Interrupt Remapping<br />
* verbose: In Xen 4.0.0 and newer, enable verbose logging while enabling IOMMU and parsing ACPI DMAR tables<br />
Usually, you just need 'iommu=1' to enable VT-d. At the same time, most of VT-d features (DMA remapping, snoop control, queued invalidation and interrupt remapping) are enabled by default if they are available. You can use 'no-xxx' to disable a feature, for example, 'iommu=no-snoop' disable snoop control.<br />
<br />
When RMRR address range is not in reserved memory (BIOS issue), can use 'iommu_inclusive_mapping=1' to work around it.<br />
<br />
The grub configuration is like:<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
== Binding Devices to pciback ==<br />
In order to pass-through devices using VT-d then need to be bound to pciback to ensure that they are not bound to another dom0 driver and this free for use by pass-through. It is possible to view this as hiding the device from Dom0.<br />
<br />
=== Binding at Boot-Time with "old-style" Xen 2.6.18 dom0 Linux kernel ===<br />
If pciback is statically compiled into the kernel, then perhaps the simplest to make PCI devices available for VT-d pass-through is to use the pciback.hide kernel parameter, as illustrated in the following grub configuration snippet. .<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/ ro pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
The arguments to pciback.hide are PCI functions in Self:BDFNotation, enclosed in brackets. The output of the lspci command may be useful when selecting PCI functions to hide.<br />
<br />
=== Binding at Boot-Time with pvops dom0 Linux kernel (2.6.31, 2.6.32, and newer) ===<br />
<br />
<pre><br />
title Xen-Linux (2.6.32.9)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.32.9 root=LABEL=/ ro xen-pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.32.9.img<br />
</pre><br />
<br />
The pciback driver name was changed from "pciback" to "xen-pciback" in December 2009 in pvops kernels.<br />
<br />
=== Binding at Run-Time ===<br />
As discussed in [[Assign_hardware_to_DomU_with_PCIBack_as_module]], it is possible to bind a device to pciback after dom0 has booted, even if it has been bound to another device in the mean time. This method can be used both when pciback is statically compiled into the dom0 kernel and when it has been compiled as a module.<br />
<br />
=== Binding at Module Insertion Time ===<br />
As also discussed in Self:Assign_hardware_to_DomU_with_PCIBack_as_module, it is possible to bind a device to pciback at the time that the pciback module is inserted. This technique is only useful if pciback has been compiled as a module.<br />
<br />
== Binding Devices to pci-stub ==<br />
If using pv-ops dom0, also can use pci-stub to hide devices for assignment (example PCI device 01:00.0) * lspci -n * locate the entry for device 01:00.0 and note down the vendor & device ID 8086:10b9 .<br />
<br />
Note: Binding devices with pci-stub only works for pci passthrough to HVM guests! Hiding with xen-pciback as mentioned above works for both HVM and PV guests and is preferred.<br />
<br />
<br />
<pre><br />
...<br />
01:00.0 0200: 8086:10b9 (rev 06)<br />
...<br />
</pre><br />
<br />
* then use following commands to hide it: .<br />
<br />
<br />
<pre><br />
echo "8086 10b9" > /sys/bus/pci/drivers/pci-stub/new_id<br />
echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind<br />
echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind<br />
</pre><br />
<br />
== Viewing Devices that are Available for Pass-Through ==<br />
xm's pci-list-assignable-devices can be used to list devices that are available for pass-through. That is, devices that have been hidden from dom0 by pciback.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list-assignable-devices<br />
0000:01:00.0<br />
0000:00:02.0<br />
</pre><br />
<br />
The devices are shown in Extended BDF notation, for more information see Self:BDFNotation.<br />
<br />
=== Implementation Details ===<br />
xm finds all devices the devices owned by pciback, checks if they have proper FLR method, checks if they have page-aligned MMIO BARs, and checks if the devices have been already assigned, finally it prints out the assignable devices. xm assumes the integrated devices (whose bus number is 0) have a proper FLR method.<br />
<br />
=== Availability of this Feature ===<br />
* Release History: Available from Xen 3.3.0<br />
* Development History: Available from [http://xenbits.xensource.com/xen-unstable.hg?rev/39c2cab9e765 changeset: 18047:39c2cab9e765], however some subsequent improvements were made and it is recommended to use [http://xenbits.xensource.com/xen-unstable.hg?rev/13690b68fd46 changeset: 18149:13690b68fd46] or newer.<br />
== Boot-Time VT-d Device Pass-Through ==<br />
As the name suggests, boot-time pass-through occurs at HVM domU (guest) boot-time. It is also referred to as ''static VT-d device pass-through'', however, this name is confusing as devices attached using this method may subsequently be hot-unplugged. To configure static pass-through for a domain add a "pci" line to the configuration file for the domain. For example, the following may be added to /etc/xen/hvm.conf:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0' ]<br />
</pre><br />
<br />
After updating the configuration file, start HVM guest and use "lspci" to verify that the the pass-through device is present in the guest. "ifconfig" or "ip addr show" may be used to see if IP address has been assigned to NIC devices.<br />
<br />
=== Virtual Slot Designation ===<br />
As of Xen 3.4.0 it is possible to specify the virtual slot that is used for by boot-time VT-d device pass-through. This is optional and when done is specified on a per-device basis by appending @SLOT_NUMBER to the BDF. For example, the following assigns 00:02.0 to virtual slot 7 while leaving xen to choose the slot for 01:00.0:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0@7' ]<br />
</pre><br />
<br />
== Viewing Devices That Have Been Passed-Through ==<br />
xm's pci-list command can be used to view the VT-d devices that have been passed-through to a domain. For example, below the devices attached to the HVM domain named HVMDomainVtd are shown.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list HVMDomainVtd<br />
VSlt domain bus slot func<br />
0x6 0x0 0x01 0x00 0x0<br />
0x7 0x0 0x00 0x02 0x0<br />
</pre><br />
<br />
Note that the domain field in the output above refers to the '''PCI domain not the Xen domain'''. PCI domains are a physical property of the host, as are PCI buses.<br />
<br />
== VT-d Device Hot-Plug ==<br />
VT-d device hot-plug involves attaching a device to an HVM domain once it has booted.<br />
<br />
xm's pci-attach command is used to perform hot-plug The device to be attached is specified using Self:BDFNotation and optionaly the desired virtual slot. If the virtual slot is omitted then a free one will be used. For example, the following command inserts the physical device into the HVM domain called HVMDomainVtd at virtual slot 7:<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7<br />
</pre><br />
<br />
As described above, xm pci-list can be used to verify the that the device is attached.<br />
<br />
== VT-d Device Hot-Unplug ==<br />
VT-d hot-unplug refers to detaching a pass-through device from a running HVM domain. The device may have been attached using boot-time VT-d device pass-through or VT-d device hot-plug.<br />
<br />
xm's pci-detach command is used to perform hot-unplug. The device to be detached is specified using Self:BDFNotation. The following command removes the device 0:2:0.0 from the domain named HVMDomainVtd.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0<br />
</pre><br />
<br />
As previously shown, pci-list can be used to show the devices that are attached to a domain and thus confirm that 0:2:0.0 has been detached as requested.<br />
<br />
== VT-d Device Hot-Plug/Unplug Usage Model ==<br />
* Live Migration: VT-d pass-through devices break live migration as physical device can't be save/restored. However, by hot-unplugging all the VT-d devices before live migration this problem is overcome.<br />
* Device Switching: VT-d hot-plug can be used to dynamically switch physical device between different HVM guest without shutdown.<br />
== Virtual Slots ==<br />
Up until Xen 3.4.0, slots 6 and 7 were reserved in HVM guests for VT-d pass-through. This meant that:<br />
<br />
* Only two pass-through devices could be attached<br />
* Attached devices would always be in slot 6 or 7<br />
* Slots 6 and 7 could not be used for IOEMU devices<br />
Xen 3.4.0 removes the reservation system. Any slot which isn't in use may be used for pass-through or IOEMU devices. Given that HVM domains have one virtual PCI bus, that a PCI bus has 32 slots for devices, and that a domain typically has a minimum of 3 IOEMU devices, 28 will be available for pass-through or extra IOEMU devices.<br />
<br />
== Enabling MSI/MSI-X for Assigned Devices ==<br />
* As of Xen 3.4.0 MSI/MSI-X is always on.<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/65dc37be0443 changeset: 18454:65dc37be0443]<br />
* In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1".<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/89d05940cc1c changeset: 18127:89d05940cc1c]<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi=1<br />
</pre><br />
<br />
* Prior to Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter" msi_irq_enable=1"<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi_irq_enable=1<br />
</pre><br />
<br />
== Single-Function and Multi-Function Pass-Through ==<br />
A PCI device may have up to 8 functions. A device with only one function is called a single-function device and a device with more than one function is called a multi-function device. A common example of a multi-function device are USB devices. For example, the following USB device is device 1a on bus 0 and has 3 functions: 0, 1 and 7.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# lspci -s 00:1a<br />
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)<br />
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)<br />
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)<br />
</pre><br />
<br />
As of Xen 3.4.1, and in all prior versions of Xen that supported VT-d pass-through, only single-function VT-d pass-through devices are supported in HVM domains. This means that if a function of as multi-function device is passed-through to an HVM domain, it will appear as function zero of a single-function device in the HVM domain, regardless of its physical function number.<br />
<br />
Single-function devices always have function zero present (and no other functions) and thus the function number does not change when a single-function device is passed through they are passed-through.<br />
<br />
The following table shows that regardless of the physical function number, the virtual function number will be zero.<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
| <br />
| Physical (dom0) <br />
|-<br />
| Single-Function <br />
| 00:02.0 <br />
|-<br />
| Multi-Function <br />
| 00:1a.1 <br />
|}<br />
<br />
Support for passing-through multi-function devices as multi-function devices has been merged into xen-unstable and should appear in 4.0. A back-port to 3.4.1 is available as per the [http://marc.info/?l=xen-devel&m=124748015304566&w=4 announcement email].<br />
<br />
This feature is accessed using extensions to BDF notation to allow a multi-function device to be specified as a single unit. For details please see Self:BDFNotation. Note that the back-port to 3.4.1 does not support the explicit mapping of virtual function numbers (the = syntax) as this is considered too green at this time.<br />
<br />
[[Category:Xen]]<br />
[[Category:HowTo]]<br />
[[Category:Users]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=VTd_HowTo&diff=7397VTd HowTo2013-03-29T19:09:41Z<p>Powerhouse: /* VT-d Enabled Systems */</p>
<hr />
<div><!-- MoinMoin name: VTdHowTo --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: VTdHowTo --><br />
<!-- Page revision: 00000104 --><br />
<!-- Original date: Sat Jul 30 03:13:48 2011 (1311995628000000) --><br />
<br />
{{TODO|This page is outdated. In particular the hardware sections need to be updated.}}<br />
<br />
__NOTOC__<br />
<!-- ## Please edit system and help pages ONLY in the moinmaster wiki! For more --><br />
<!-- ## information, please see [[MoinMaster]]:[[MoinPagesEditorGroup]]. --><br />
<!-- ##acl [[MoinPagesEditorGroup]]:read,write,delete,revert All:read --><br />
<!-- ##master-page:[[HelpTemplate]] --><br />
<!-- ##master-date:Unknown-Date --><br />
<!-- #format wiki --><br />
<!-- #language en --><br />
Please also check [[XenPCIpassthrough]] wiki page for more general information about Xen PCI passthru usage!<br />
<br />
VT-d Pass-Through is a technique to give a domU exclusive access to a PCI function using the IOMMU provided by VT-d. It is primarily targeted at HVM (fully virtualised) guests because PV (paravirtualized) pass-through does not require VT-d (altough it may be utilized too)<br />
<br />
== Xen 4.1 xl tools notes ==<br />
* Only devices with FLR capabilities are supported.<br />
* Passing through a PCI card without FLR capability will result an error.<br />
<br />
To check if your PCI devices have FLR function, check in this wiki, [[http://wiki.xen.org/wiki/Xen_PCI_Passthrough#How_can_I_check_if_PCI_device_supports_FLR_.28Function_Level_Reset.29_.3F How can I check if PCI device supports FLR (Function Level Reset) ?]]<br />
<br />
If you see output with "FLReset-" then your PCI device don't support FLR function. If output have "FLReset+" then it does.<br />
<br />
As this time of writing, June 2012, there are very very few PCI devices support FLR function.<br />
<br />
== Supported OSes ==<br />
* Dom0 (Host) OS: PAE, 64-bit<br />
* DomU (Guest) OS: 32-bit, PAE, 64-bit<br />
=== Tested DomO OSes ===<br />
* 64-bit host: 32/PAE/64 Linux/XP/Win2003/Vista guests<br />
* PAE host: 32/PAE Linux/XP/Win2003/Vista guests<br />
== VT-d Enabled Systems ==<br />
Note that in addition to the motherboard chipset and BIOS also your CPU must have support for IOMMU IO virtualization (VT-d)! Make sure to check your CPU vendors spec sheets for more information.<br />
<br />
CPUs known to work (with a motherboard from below list and a proper BIOS):<br />
<br />
(Intel reference the list of [http://ark.intel.com/search/advanced/?s=t&VTD=true compatible processors vt-d])<br />
<br />
* Intel Core2Duo (with VT-x)<br />
* Intel Core2Quad<br />
* Intel Core i7<br />
* Intel Core i5 (vPro Brand)<br />
* AMD FX-8120 / FX-8150<br />
For VT-d enabling work on Xen, we have been using development systems using following Intel motherboards:<br />
<br />
* DQ35MP<br />
* DQ35JO<br />
* Notes on VT-d compatibility:<br />
** VT-d is enabled on the following chipsets:<br />
*** Intel Q35 (desktop / workstation)<br />
*** Intel Q45 (desktop / workstation)<br />
*** Intel X58 (desktop / workstation)<br />
*** Intel X79<br />
*** Intel 55x0 (server)<br />
*** Intel 3450 (workstation / server)<br />
* The following chipsets have VT-d capability in theory, but most OEMs (such as Asus and Gigabyte) do not have it enabled on boards based on these:<br />
** Intel X38 (desktop / workstation)<br />
** Intel X48 (desktop / workstation)<br />
** Intel 32x0 (server)<br />
* For Intel Desktop Boards, these have VT-d support enabled:<br />
** Intel DQ35JO<br />
** Intel DQ35MP<br />
** Intel DX38BT, DX48BT2 (BIOS 1554 or 1782 required, VT-d is known to not work in other versions; iommu=workaround_bios_bug required) <br />
** Intel DQ45CB (BIOS 0061 required, previous versions are known to cause problems)<br />
** Intel DQ45EK<br />
** Intel DX58SO<br />
** Intel DQ67SW<br />
* For ASUS Desktop Boards, these have VT-d support enabled, but Asus does NOT support Linux, so you are on your own with any Linux or Xen issues like broken BIOSes:<br />
** ASUS P5E-VM DO (Intel Q35 chipset) requires IGD to be enabled (otherwise DMAR-table becomes corrupted)<br />
** ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS update to correct DMAR-table issue<br />
** ASUS P6T documentation claims support, but does not work due to DMAR-table issue<br />
** ASUS P6T6 WS Revolution (Intel X58 chipset) incorrect bios DMAR-table<br />
** ASUS Sabertooth X58 (Intel X58 chipset) incorrect bios DMAR-table (2nd RMRR structure is incorrect)<br />
** ASUS Sabertooth X79 (with BIOS release 1203 and Marvell SATA controller disabled)<br />
** ASUS Z8NA-D6 (dual processor nehalem board) works<br />
* Most server boards based on the Tylersburg chipset (55x0) and few boards based on 32x0 should have working VT-d, known examples are:<br />
** Intel server board: [http://www.intel.com/support/motherboards/server/S3200SH/sb/CS-028345.htm S3210SHLX] (BIOS >R0044 required.)<br />
** Supermicro server mainboard: [http://www.supermicro.com/products/motherboard/QPI/5500/X8DT3-F.cfm X8DT3-F]<br />
** Supermicro server 5026T-TB: [http://www.supermicro.com/products/system/2U/5026/SYS-5026T-T.cfm] Only work USB 1.1.<br />
* These motherboards are known to have broken BIOS preventing IO virtualization (VT-d IOMMU) from working:<br />
** Supermicro X7SB4 (with official BIOS 1.2a) has broken ACPI DMAR table with zero length entries. BIOS version 1.3 Beta fixes the problem.<br />
** Samsung X460 laptop: BIOS doesn't provide DMAR table so VT-d cannot be used.<br />
* As far as we know, following OEM systems also have VT-d enabled. Feel free to add others as they become available.<br />
** Dell: [http://www.dell.com/content/products/category.aspx/optix?c=us&cs=555&l=en&s=biz Optiplex 755], [http://www.dell.com/us/business/p/optiplex-980/pd Optiplex 980]<br />
** HP Compaq: [http://h10010.www1.hp.com/wwpc/us/en/en/WF04a/12454-12454-64287-321860-3328898.html DC7800]<br />
** Fujitsu-Siemens: [http://facts.fujitsu-siemens.com/index.cfm?fuseaction=dspcontent&lid=134&page=3 Esprimo 5925]<br />
** Lenovo: [http://shop.lenovo.com/us/workstations/thinkstation/workstation/e20 ThinkStation E20] and Thinkserver TS200v<br />
** HP: [http://www.hp.com/united-states/campaigns/workstations/z420_features.html#.UG6055gxrug Z420 Workstation] <br />
*VT-d Compatible ASRock Motherboards (Socket 1155 and 2011) probably all Z68 and Z77 board's<br />
**ASRock H77 Pro4-M<br />
**ASRock Z68 Extreme4 Gen3<br />
**ASRock Z68 Professional Gen3<br />
**ASRock Z77 Pro3<br />
**ASRock Z77 Pro4<br />
**ASRock Z77 Extreme4<br />
**ASRock Z77 Extreme6<br />
**ASRock Z77 Professional<br />
**ASRock X79 Extreme9 (socket 2011)<br />
<br />
For Z77 Extreme4/Extreme6 users: ASRock introduced change in how VT-d setting can be modified within BIOS settings, this affects BIOS version >2.50 (for Extreme4 see [http://www.asrock.com/mb/Intel/Z77%20Extreme4/index.us.asp?cat=Download&os=BIOS]) and >2.30 (for Extreme 6 see [http://www.asrock.com/mb/Intel/Z77%20Extreme6/index.us.asp?cat=Download&os=BIOS]). So these BIOS versions '''do not disable VT-d support''', but just disable user ability to modify this setting within BIOS e.g. for non-K processors VT-d will be always enabled and for K-series disabled.<br />
<br />
* Gigabyte Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** GA-X79-UD5 (rev. 1.0)<br />
** GA-X79-UD7 (rev. 1.0)<br />
** GA-Z77X-D3H (rev. 1.0)<br />
* MSI 1155 Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** Z68A-GD80-G3<br />
** Z77A-GD80<br />
** Z77A-GD65<br />
** Z77A-GD55<br />
** Z77A-G45<br />
** Z77A-G43<br />
<br />
== AMD desktop chipsets with IOMMU support ==<br />
* AMD 890FX chipset supports IOMMU. Other 890 chipsets don't have IOMMU support!<br />
* AMD 990FX, 990X and 970 chipsets support IOMMU.<br />
* Even when the chipset supports IOMMU, the bios must have a ACPI IVRS table to enable the use of it! So actual support depends on the motherboard manufacturer. At the time of writing all motherboards seem to have a (beta)bios available supporting the IOMMU. A thread with user expiriences can also be found at this forum: http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/<br />
* Motherboards with a BIOS supporting the IOMMU(as reported by users):<br />
** ASUS Crosshair IV (reported working by Jens Krehbiel-Gräther)<br />
** ASUS Crosshair V Formula (reported working by Pavel Matěja)<br />
** ASUS F2A85-V PRO<br />
** ASUS M4A89TD Pro/USB3 (reported working by Jens Krehbiel-Gräther)<br />
** Asrock 890FX Deluxe3 (reported working by Jens Krehbiel-Gräther)<br />
** Biostar TA890FXE (from bios version 89FAD629.BST reported working by Joop Boonen, Konrad Rzeszutek Wilk)<br />
** Gigabyte GA-970A-UD3 (Bios F7)<br />
* Motherboards with a beta-bios available from tech-support that supports the IOMMU:<br />
** Gigabyte GA-890FXA-UD5<br />
** Gigabyte GA-890FXA-UD7<br />
** MSI 890FXA-GD70 (from beta-bios 1.75 reported working by Sander Eikelenboom)<br />
<br />
== AMD server (opteron) chipsets with IOMMU support ==<br />
* AMD SR5690 / SR5670 (Tyan S8212)<br />
== Caveat on Conventional PCI Device Pass-Through ==<br />
* The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.<br />
* PCIe devices do not have this restriction.<br />
== Notes on Compiling Source Code ==<br />
It is not the intention of this document to describe in detail how to compile and install Xen from source. Rather, the intention of this section is to highlight points of particular note when compiling from source for use with VT-d.<br />
<br />
=== Dom0 Kernel ===<br />
A prerequisite for binding devices to pciback, which is discussed in more detail below, is that pciback is compiled statically into or as a module for the dom0 kernel. This means that in the kernel config CONFIG_XEN_PCIDEV_BACKEND=y should be set to statically compile pciback into the kernel or CONFIG_XEN_PCIDEV_BACKEND=m should be set to compile pciback as a module.<br />
<br />
For pv-ops dom0, you can use pci-stub to hide device. Set CONFIG_PCI_STUB=y to build it into kernel or CONFIG_PCI_STUB=m to build as a module.<br />
<br />
=== Device Model ===<br />
When building the device model, also known as qemu-dm and qemu-xen, often automatically downloaded during compilation and found in the tools/ioemu-dir/ subdirectory of the xen tree, it is important that the development headers and libraries for libpci are installed. Otherwise qemu-dm will be built without pass-through support.<br />
<br />
If the device model has been compiled without pass-through support, the following error will show up in xend.log at run-time when pass-through is attempted.<br />
<br />
* <br />
<pre><br />
ERROR (XendDomainInfo:581) Device model didn't tell the vslots for PCI device<br />
</pre><br />
<br />
== VT-d boot parameter: iommu ==<br />
VT-d is disabled by default, to enable it, need 'iommu' parameter to enable it.<br />
<br />
* off|no|false|disable: Disable IOMMU (default)<br />
* pv: Enable IOMMU for PV domains<br />
* no-pv: Disable IOMMU for PV domains (default)<br />
* force|required: Don't boot unless IOMMU is enabled<br />
* workaround_bios_bug: Workaround some bios issues to still enable VT-d, don't guarantee security<br />
* pass-through: Enable VT-d DMA pass-through (no DMA translation for Dom0)<br />
* no-snoop: Disable VT-d Snoop Control<br />
* no-qinval: Disable VT-d Queued Invalidation<br />
* no-intremap: Disable VT-d Interrupt Remapping<br />
* verbose: In Xen 4.0.0 and newer, enable verbose logging while enabling IOMMU and parsing ACPI DMAR tables<br />
Usually, you just need 'iommu=1' to enable VT-d. At the same time, most of VT-d features (DMA remapping, snoop control, queued invalidation and interrupt remapping) are enabled by default if they are available. You can use 'no-xxx' to disable a feature, for example, 'iommu=no-snoop' disable snoop control.<br />
<br />
When RMRR address range is not in reserved memory (BIOS issue), can use 'iommu_inclusive_mapping=1' to work around it.<br />
<br />
The grub configuration is like:<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
== Binding Devices to pciback ==<br />
In order to pass-through devices using VT-d then need to be bound to pciback to ensure that they are not bound to another dom0 driver and this free for use by pass-through. It is possible to view this as hiding the device from Dom0.<br />
<br />
=== Binding at Boot-Time with "old-style" Xen 2.6.18 dom0 Linux kernel ===<br />
If pciback is statically compiled into the kernel, then perhaps the simplest to make PCI devices available for VT-d pass-through is to use the pciback.hide kernel parameter, as illustrated in the following grub configuration snippet. .<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/ ro pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
The arguments to pciback.hide are PCI functions in Self:BDFNotation, enclosed in brackets. The output of the lspci command may be useful when selecting PCI functions to hide.<br />
<br />
=== Binding at Boot-Time with pvops dom0 Linux kernel (2.6.31, 2.6.32, and newer) ===<br />
<br />
<pre><br />
title Xen-Linux (2.6.32.9)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.32.9 root=LABEL=/ ro xen-pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.32.9.img<br />
</pre><br />
<br />
The pciback driver name was changed from "pciback" to "xen-pciback" in December 2009 in pvops kernels.<br />
<br />
=== Binding at Run-Time ===<br />
As discussed in [[Assign_hardware_to_DomU_with_PCIBack_as_module]], it is possible to bind a device to pciback after dom0 has booted, even if it has been bound to another device in the mean time. This method can be used both when pciback is statically compiled into the dom0 kernel and when it has been compiled as a module.<br />
<br />
=== Binding at Module Insertion Time ===<br />
As also discussed in Self:Assign_hardware_to_DomU_with_PCIBack_as_module, it is possible to bind a device to pciback at the time that the pciback module is inserted. This technique is only useful if pciback has been compiled as a module.<br />
<br />
== Binding Devices to pci-stub ==<br />
If using pv-ops dom0, also can use pci-stub to hide devices for assignment (example PCI device 01:00.0) * lspci -n * locate the entry for device 01:00.0 and note down the vendor & device ID 8086:10b9 .<br />
<br />
Note: Binding devices with pci-stub only works for pci passthrough to HVM guests! Hiding with xen-pciback as mentioned above works for both HVM and PV guests and is preferred.<br />
<br />
<br />
<pre><br />
...<br />
01:00.0 0200: 8086:10b9 (rev 06)<br />
...<br />
</pre><br />
<br />
* then use following commands to hide it: .<br />
<br />
<br />
<pre><br />
echo "8086 10b9" > /sys/bus/pci/drivers/pci-stub/new_id<br />
echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind<br />
echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind<br />
</pre><br />
<br />
== Viewing Devices that are Available for Pass-Through ==<br />
xm's pci-list-assignable-devices can be used to list devices that are available for pass-through. That is, devices that have been hidden from dom0 by pciback.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list-assignable-devices<br />
0000:01:00.0<br />
0000:00:02.0<br />
</pre><br />
<br />
The devices are shown in Extended BDF notation, for more information see Self:BDFNotation.<br />
<br />
=== Implementation Details ===<br />
xm finds all devices the devices owned by pciback, checks if they have proper FLR method, checks if they have page-aligned MMIO BARs, and checks if the devices have been already assigned, finally it prints out the assignable devices. xm assumes the integrated devices (whose bus number is 0) have a proper FLR method.<br />
<br />
=== Availability of this Feature ===<br />
* Release History: Available from Xen 3.3.0<br />
* Development History: Available from [http://xenbits.xensource.com/xen-unstable.hg?rev/39c2cab9e765 changeset: 18047:39c2cab9e765], however some subsequent improvements were made and it is recommended to use [http://xenbits.xensource.com/xen-unstable.hg?rev/13690b68fd46 changeset: 18149:13690b68fd46] or newer.<br />
== Boot-Time VT-d Device Pass-Through ==<br />
As the name suggests, boot-time pass-through occurs at HVM domU (guest) boot-time. It is also referred to as ''static VT-d device pass-through'', however, this name is confusing as devices attached using this method may subsequently be hot-unplugged. To configure static pass-through for a domain add a "pci" line to the configuration file for the domain. For example, the following may be added to /etc/xen/hvm.conf:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0' ]<br />
</pre><br />
<br />
After updating the configuration file, start HVM guest and use "lspci" to verify that the the pass-through device is present in the guest. "ifconfig" or "ip addr show" may be used to see if IP address has been assigned to NIC devices.<br />
<br />
=== Virtual Slot Designation ===<br />
As of Xen 3.4.0 it is possible to specify the virtual slot that is used for by boot-time VT-d device pass-through. This is optional and when done is specified on a per-device basis by appending @SLOT_NUMBER to the BDF. For example, the following assigns 00:02.0 to virtual slot 7 while leaving xen to choose the slot for 01:00.0:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0@7' ]<br />
</pre><br />
<br />
== Viewing Devices That Have Been Passed-Through ==<br />
xm's pci-list command can be used to view the VT-d devices that have been passed-through to a domain. For example, below the devices attached to the HVM domain named HVMDomainVtd are shown.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list HVMDomainVtd<br />
VSlt domain bus slot func<br />
0x6 0x0 0x01 0x00 0x0<br />
0x7 0x0 0x00 0x02 0x0<br />
</pre><br />
<br />
Note that the domain field in the output above refers to the '''PCI domain not the Xen domain'''. PCI domains are a physical property of the host, as are PCI buses.<br />
<br />
== VT-d Device Hot-Plug ==<br />
VT-d device hot-plug involves attaching a device to an HVM domain once it has booted.<br />
<br />
xm's pci-attach command is used to perform hot-plug The device to be attached is specified using Self:BDFNotation and optionaly the desired virtual slot. If the virtual slot is omitted then a free one will be used. For example, the following command inserts the physical device into the HVM domain called HVMDomainVtd at virtual slot 7:<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7<br />
</pre><br />
<br />
As described above, xm pci-list can be used to verify the that the device is attached.<br />
<br />
== VT-d Device Hot-Unplug ==<br />
VT-d hot-unplug refers to detaching a pass-through device from a running HVM domain. The device may have been attached using boot-time VT-d device pass-through or VT-d device hot-plug.<br />
<br />
xm's pci-detach command is used to perform hot-unplug. The device to be detached is specified using Self:BDFNotation. The following command removes the device 0:2:0.0 from the domain named HVMDomainVtd.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0<br />
</pre><br />
<br />
As previously shown, pci-list can be used to show the devices that are attached to a domain and thus confirm that 0:2:0.0 has been detached as requested.<br />
<br />
== VT-d Device Hot-Plug/Unplug Usage Model ==<br />
* Live Migration: VT-d pass-through devices break live migration as physical device can't be save/restored. However, by hot-unplugging all the VT-d devices before live migration this problem is overcome.<br />
* Device Switching: VT-d hot-plug can be used to dynamically switch physical device between different HVM guest without shutdown.<br />
== Virtual Slots ==<br />
Up until Xen 3.4.0, slots 6 and 7 were reserved in HVM guests for VT-d pass-through. This meant that:<br />
<br />
* Only two pass-through devices could be attached<br />
* Attached devices would always be in slot 6 or 7<br />
* Slots 6 and 7 could not be used for IOEMU devices<br />
Xen 3.4.0 removes the reservation system. Any slot which isn't in use may be used for pass-through or IOEMU devices. Given that HVM domains have one virtual PCI bus, that a PCI bus has 32 slots for devices, and that a domain typically has a minimum of 3 IOEMU devices, 28 will be available for pass-through or extra IOEMU devices.<br />
<br />
== Enabling MSI/MSI-X for Assigned Devices ==<br />
* As of Xen 3.4.0 MSI/MSI-X is always on.<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/65dc37be0443 changeset: 18454:65dc37be0443]<br />
* In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1".<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/89d05940cc1c changeset: 18127:89d05940cc1c]<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi=1<br />
</pre><br />
<br />
* Prior to Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter" msi_irq_enable=1"<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi_irq_enable=1<br />
</pre><br />
<br />
== Single-Function and Multi-Function Pass-Through ==<br />
A PCI device may have up to 8 functions. A device with only one function is called a single-function device and a device with more than one function is called a multi-function device. A common example of a multi-function device are USB devices. For example, the following USB device is device 1a on bus 0 and has 3 functions: 0, 1 and 7.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# lspci -s 00:1a<br />
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)<br />
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)<br />
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)<br />
</pre><br />
<br />
As of Xen 3.4.1, and in all prior versions of Xen that supported VT-d pass-through, only single-function VT-d pass-through devices are supported in HVM domains. This means that if a function of as multi-function device is passed-through to an HVM domain, it will appear as function zero of a single-function device in the HVM domain, regardless of its physical function number.<br />
<br />
Single-function devices always have function zero present (and no other functions) and thus the function number does not change when a single-function device is passed through they are passed-through.<br />
<br />
The following table shows that regardless of the physical function number, the virtual function number will be zero.<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
| <br />
| Physical (dom0) <br />
|-<br />
| Single-Function <br />
| 00:02.0 <br />
|-<br />
| Multi-Function <br />
| 00:1a.1 <br />
|}<br />
<br />
Support for passing-through multi-function devices as multi-function devices has been merged into xen-unstable and should appear in 4.0. A back-port to 3.4.1 is available as per the [http://marc.info/?l=xen-devel&m=124748015304566&w=4 announcement email].<br />
<br />
This feature is accessed using extensions to BDF notation to allow a multi-function device to be specified as a single unit. For details please see Self:BDFNotation. Note that the back-port to 3.4.1 does not support the explicit mapping of virtual function numbers (the = syntax) as this is considered too green at this time.<br />
<br />
[[Category:Xen]]<br />
[[Category:HowTo]]<br />
[[Category:Users]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=VTd_HowTo&diff=5950VTd HowTo2013-01-18T18:22:48Z<p>Powerhouse: /* VT-d Enabled Systems */</p>
<hr />
<div><!-- MoinMoin name: VTdHowTo --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: VTdHowTo --><br />
<!-- Page revision: 00000104 --><br />
<!-- Original date: Sat Jul 30 03:13:48 2011 (1311995628000000) --><br />
<br />
{{TODO|This page is outdated. In particular the hardware sections need to be updated.}}<br />
<br />
__NOTOC__<br />
<!-- ## Please edit system and help pages ONLY in the moinmaster wiki! For more --><br />
<!-- ## information, please see [[MoinMaster]]:[[MoinPagesEditorGroup]]. --><br />
<!-- ##acl [[MoinPagesEditorGroup]]:read,write,delete,revert All:read --><br />
<!-- ##master-page:[[HelpTemplate]] --><br />
<!-- ##master-date:Unknown-Date --><br />
<!-- #format wiki --><br />
<!-- #language en --><br />
Please also check [[XenPCIpassthrough]] wiki page for more general information about Xen PCI passthru usage!<br />
<br />
VT-d Pass-Through is a technique to give a domU exclusive access to a PCI function using the IOMMU provided by VT-d. It is primarily targeted at HVM (fully virtualised) guests because PV (paravirtualized) pass-through does not require VT-d (altough it may be utilized too)<br />
<br />
== Xen 4.1 xl tools notes ==<br />
* Only devices with FLR capabilities are supported.<br />
* Passing through a PCI card without FLR capability will result an error.<br />
<br />
To check if your PCI devices have FLR function, check in this wiki, [[http://wiki.xen.org/wiki/Xen_PCI_Passthrough#How_can_I_check_if_PCI_device_supports_FLR_.28Function_Level_Reset.29_.3F How can I check if PCI device supports FLR (Function Level Reset) ?]]<br />
<br />
If you see output with "FLReset-" then your PCI device don't support FLR function. If output have "FLReset+" then it does.<br />
<br />
As this time of writing, June 2012, there are very very few PCI devices support FLR function.<br />
<br />
== Supported OSes ==<br />
* Dom0 (Host) OS: PAE, 64-bit<br />
* DomU (Guest) OS: 32-bit, PAE, 64-bit<br />
=== Tested DomO OSes ===<br />
* 64-bit host: 32/PAE/64 Linux/XP/Win2003/Vista guests<br />
* PAE host: 32/PAE Linux/XP/Win2003/Vista guests<br />
== VT-d Enabled Systems ==<br />
Note that in addition to the motherboard chipset and BIOS also your CPU must have support for IOMMU IO virtualization (VT-d)! Make sure to check your CPU vendors spec sheets for more information.<br />
<br />
CPUs known to work (with a motherboard from below list and a proper BIOS):<br />
<br />
(Intel reference the list of [http://ark.intel.com/search/advanced/?s=t&VTD=true compatible processors vt-d])<br />
<br />
* Intel [[Core2Duo]] (with VT-x)<br />
* Intel [[Core2Quad]]<br />
* Intel Core i7<br />
* Intel Core i5 (vPro Brand)<br />
* AMD FX-8120 / FX-8150<br />
For VT-d enabling work on Xen, we have been using development systems using following Intel motherboards:<br />
<br />
* DQ35MP<br />
* DQ35JO<br />
* Notes on VT-d compatibility:<br />
** VT-d is enabled on the following chipsets:<br />
*** Intel Q35 (desktop / workstation)<br />
*** Intel Q45 (desktop / workstation)<br />
*** Intel X58 (desktop / workstation)<br />
*** Intel X79<br />
*** Intel 55x0 (server)<br />
*** Intel 3450 (workstation / server)<br />
* The following chipsets have VT-d capability in theory, but most OEMs (such as Asus and Gigabyte) do not have it enabled on boards based on these:<br />
** Intel X38 (desktop / workstation)<br />
** Intel X48 (desktop / workstation)<br />
** Intel 32x0 (server)<br />
* For Intel Desktop Boards, these have VT-d support enabled:<br />
** Intel DQ35JO<br />
** Intel DQ35MP<br />
** Intel DX38BT, DX48BT2 (BIOS 1554 or 1782 required, VT-d is known to not work in other versions; iommu=workaround_bios_bug required) <br />
** Intel DQ45CB (BIOS 0061 required, previous versions are known to cause problems)<br />
** Intel DQ45EK<br />
** Intel DX58SO<br />
** Intel DQ67SW<br />
* For ASUS Desktop Boards, these have VT-d support enabled, but Asus does NOT support Linux, so you are on your own with any Linux or Xen issues like broken BIOSes:<br />
** ASUS P5E-VM DO (Intel Q35 chipset) requires IGD to be enabled (otherwise DMAR-table becomes corrupted)<br />
** ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS update to correct DMAR-table issue<br />
** ASUS [[P6T6]] WS Revolution (Intel X58 chipset) incorrect bios DMAR-table<br />
** ASUS Sabertooth X58 (Intel X58 chipset) incorrect bios DMAR-table (2nd RMRR structure is incorrect)<br />
** ASUS Z8NA-D6 (dual processor nehalem board) works<br />
* Most server boards based on the Tylersburg chipset (55x0) and few boards based on 32x0 should have working VT-d, known examples are:<br />
** Intel server board: [http://www.intel.com/support/motherboards/server/S3200SH/sb/CS-028345.htm S3210SHLX] (BIOS >R0044 required.)<br />
** Supermicro server mainboard: [http://www.supermicro.com/products/motherboard/QPI/5500/X8DT3-F.cfm X8DT3-F]<br />
** Supermicro server 5026T-TB: [http://www.supermicro.com/products/system/2U/5026/SYS-5026T-T.cfm] Only work USB 1.1.<br />
* These motherboards are known to have broken BIOS preventing IO virtualization (VT-d IOMMU) from working:<br />
** Supermicro X7SB4 (with official BIOS 1.2a) has broken ACPI DMAR table with zero length entries. BIOS version 1.3 Beta fixes the problem.<br />
** Samsung X460 laptop: BIOS doesn't provide DMAR table so VT-d cannot be used.<br />
* As far as we know, following OEM systems also have VT-d enabled. Feel free to add others as they become available.<br />
** Dell: [http://www.dell.com/content/products/category.aspx/optix?c=us&cs=555&l=en&s=biz Optiplex 755], [http://www.dell.com/us/business/p/optiplex-980/pd Optiplex 980]<br />
** HP Compaq: [http://h10010.www1.hp.com/wwpc/us/en/en/WF04a/12454-12454-64287-321860-3328898.html DC7800]<br />
** Fujitsu-Siemens: [http://facts.fujitsu-siemens.com/index.cfm?fuseaction=dspcontent&lid=134&page=3 Esprimo 5925]<br />
** Lenovo: [http://shop.lenovo.com/us/workstations/thinkstation/workstation/e20 ThinkStation E20] and Thinkserver TS200v<br />
** HP: [http://www.hp.com/united-states/campaigns/workstations/z420_features.html#.UG6055gxrug Z420 Workstation] <br />
*VT-d Compatible ASRock Motherboards (Socket 1155 and 2011) probably all Z68 and Z77 board's<br />
**ASRock H77 Pro4-M<br />
**ASRock Z68 Extreme4 Gen3<br />
**ASRock Z68 Professional Gen3<br />
**ASRock Z77 Pro3<br />
**ASRock Z77 Extreme4 up to BIOS 2.50 (starting from BIOS release 2.70 there is no VT-d support !!!), see [http://www.asrock.com/mb/Intel/Z77%20Extreme4/index.us.asp?cat=Download&os=BIOS] and [http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine/480#post_19078253]<br />
**ASRock Z77 Extreme6 up to BIOS 2.30 (starting from BIOS release 2.40 there is no VT-d support !!!), see [http://www.asrock.com/mb/Intel/Z77%20Extreme6/index.us.asp?cat=Download&os=BIOS] and [http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine/480#post_19078253]<br />
**ASRock Z77 Professional<br />
**ASRock X79 Extreme9 (socket 2011)<br />
* Gigabyte Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** GA-X79-UD5 (rev. 1.0)<br />
** GA-X79-UD7 (rev. 1.0)<br />
** GA-Z77X-D3H (rev. 1.0)<br />
* MSI 1155 Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** Z68A-GD80-G3<br />
** Z77A-GD80<br />
** Z77A-GD65<br />
** Z77A-GD55<br />
** Z77A-G45<br />
** Z77A-G43<br />
<br />
== AMD desktop chipsets with IOMMU support ==<br />
* AMD 890FX chipset supports IOMMU. Other 890 chipsets don't have IOMMU support!<br />
* AMD 990FX, 990X and 970 chipsets support IOMMU.<br />
* Even when the chipset supports IOMMU, the bios must have a ACPI IVRS table to enable the use of it! So actual support depends on the motherboard manufacturer. At the time of writing all motherboards seem to have a (beta)bios available supporting the IOMMU. A thread with user expiriences can also be found at this forum: http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/<br />
* Motherboards with a BIOS supporting the IOMMU(as reported by users):<br />
** ASUS Crosshair IV (reported working by Jens Krehbiel-Gräther)<br />
** ASUS Crosshair V Formula (reported working by Pavel Matěja)<br />
** ASUS F2A85-V PRO<br />
** ASUS M4A89TD Pro/USB3 (reported working by Jens Krehbiel-Gräther)<br />
** Asrock 890FX Deluxe3 (reported working by Jens Krehbiel-Gräther)<br />
** Biostar TA890FXE (from bios version 89FAD629.BST reported working by Joop Boonen, Konrad Rzeszutek Wilk)<br />
* Motherboards with a beta-bios available from tech-support that supports the IOMMU:<br />
** Gigabyte GA-890FXA-UD5<br />
** Gigabyte GA-890FXA-UD7<br />
** MSI 890FXA-GD70 (from beta-bios 1.75 reported working by Sander Eikelenboom)<br />
<br />
== AMD server (opteron) chipsets with IOMMU support ==<br />
* AMD SR5690 / SR5670 (Tyan S8212)<br />
== Caveat on Conventional PCI Device Pass-Through ==<br />
* The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.<br />
* PCIe devices do not have this restriction.<br />
== Notes on Compiling Source Code ==<br />
It is not the intention of this document to describe in detail how to compile and install Xen from source. Rather, the intention of this section is to highlight points of particular note when compiling from source for use with VT-d.<br />
<br />
=== Dom0 Kernel ===<br />
A prerequisite for binding devices to pciback, which is discussed in more detail below, is that pciback is compiled statically into or as a module for the dom0 kernel. This means that in the kernel config CONFIG_XEN_PCIDEV_BACKEND=y should be set to statically compile pciback into the kernel or CONFIG_XEN_PCIDEV_BACKEND=m should be set to compile pciback as a module.<br />
<br />
For pv-ops dom0, you can use pci-stub to hide device. Set CONFIG_PCI_STUB=y to build it into kernel or CONFIG_PCI_STUB=m to build as a module.<br />
<br />
=== Device Model ===<br />
When building the device model, also known as qemu-dm and qemu-xen, often automatically downloaded during compilation and found in the tools/ioemu-dir/ subdirectory of the xen tree, it is important that the development headers and libraries for libpci are installed. Otherwise qemu-dm will be built without pass-through support.<br />
<br />
If the device model has been compiled without pass-through support, the following error will show up in xend.log at run-time when pass-through is attempted.<br />
<br />
* <br />
<pre><br />
ERROR (XendDomainInfo:581) Device model didn't tell the vslots for PCI device<br />
</pre><br />
<br />
== VT-d boot parameter: iommu ==<br />
VT-d is disabled by default, to enable it, need 'iommu' parameter to enable it.<br />
<br />
* off|no|false|disable: Disable IOMMU (default)<br />
* pv: Enable IOMMU for PV domains<br />
* no-pv: Disable IOMMU for PV domains (default)<br />
* force|required: Don't boot unless IOMMU is enabled<br />
* workaround_bios_bug: Workaround some bios issues to still enable VT-d, don't guarantee security<br />
* pass-through: Enable VT-d DMA pass-through (no DMA translation for Dom0)<br />
* no-snoop: Disable VT-d Snoop Control<br />
* no-qinval: Disable VT-d Queued Invalidation<br />
* no-intremap: Disable VT-d Interrupt Remapping<br />
* verbose: In Xen 4.0.0 and newer, enable verbose logging while enabling IOMMU and parsing ACPI DMAR tables<br />
Usually, you just need 'iommu=1' to enable VT-d. At the same time, most of VT-d features (DMA remapping, snoop control, queued invalidation and interrupt remapping) are enabled by default if they are available. You can use 'no-xxx' to disable a feature, for example, 'iommu=no-snoop' disable snoop control.<br />
<br />
When RMRR address range is not in reserved memory (BIOS issue), can use 'iommu_inclusive_mapping=1' to work around it.<br />
<br />
The grub configuration is like:<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
== Binding Devices to pciback ==<br />
In order to pass-through devices using VT-d then need to be bound to pciback to ensure that they are not bound to another dom0 driver and this free for use by pass-through. It is possible to view this as hiding the device from Dom0.<br />
<br />
=== Binding at Boot-Time with "old-style" Xen 2.6.18 dom0 Linux kernel ===<br />
If pciback is statically compiled into the kernel, then perhaps the simplest to make PCI devices available for VT-d pass-through is to use the pciback.hide kernel parameter, as illustrated in the following grub configuration snippet. .<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/ ro pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
The arguments to pciback.hide are PCI functions in Self:BDFNotation, enclosed in brackets. The output of the lspci command may be useful when selecting PCI functions to hide.<br />
<br />
=== Binding at Boot-Time with pvops dom0 Linux kernel (2.6.31, 2.6.32, and newer) ===<br />
<br />
<pre><br />
title Xen-Linux (2.6.32.9)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.32.9 root=LABEL=/ ro xen-pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.32.9.img<br />
</pre><br />
<br />
The pciback driver name was changed from "pciback" to "xen-pciback" in December 2009 in pvops kernels.<br />
<br />
=== Binding at Run-Time ===<br />
As discussed in [[Assign_hardware_to_DomU_with_PCIBack_as_module]], it is possible to bind a device to pciback after dom0 has booted, even if it has been bound to another device in the mean time. This method can be used both when pciback is statically compiled into the dom0 kernel and when it has been compiled as a module.<br />
<br />
=== Binding at Module Insertion Time ===<br />
As also discussed in Self:Assign_hardware_to_DomU_with_PCIBack_as_module, it is possible to bind a device to pciback at the time that the pciback module is inserted. This technique is only useful if pciback has been compiled as a module.<br />
<br />
== Binding Devices to pci-stub ==<br />
If using pv-ops dom0, also can use pci-stub to hide devices for assignment (example PCI device 01:00.0) * lspci -n * locate the entry for device 01:00.0 and note down the vendor & device ID 8086:10b9 .<br />
<br />
Note: Binding devices with pci-stub only works for pci passthrough to HVM guests! Hiding with xen-pciback as mentioned above works for both HVM and PV guests and is preferred.<br />
<br />
<br />
<pre><br />
...<br />
01:00.0 0200: 8086:10b9 (rev 06)<br />
...<br />
</pre><br />
<br />
* then use following commands to hide it: .<br />
<br />
<br />
<pre><br />
echo "8086 10b9" > /sys/bus/pci/drivers/pci-stub/new_id<br />
echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind<br />
echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind<br />
</pre><br />
<br />
== Viewing Devices that are Available for Pass-Through ==<br />
xm's pci-list-assignable-devices can be used to list devices that are available for pass-through. That is, devices that have been hidden from dom0 by pciback.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list-assignable-devices<br />
0000:01:00.0<br />
0000:00:02.0<br />
</pre><br />
<br />
The devices are shown in Extended BDF notation, for more information see Self:BDFNotation.<br />
<br />
=== Implementation Details ===<br />
xm finds all devices the devices owned by pciback, checks if they have proper FLR method, checks if they have page-aligned MMIO BARs, and checks if the devices have been already assigned, finally it prints out the assignable devices. xm assumes the integrated devices (whose bus number is 0) have a proper FLR method.<br />
<br />
=== Availability of this Feature ===<br />
* Release History: Available from Xen 3.3.0<br />
* Development History: Available from [http://xenbits.xensource.com/xen-unstable.hg?rev/39c2cab9e765 changeset: 18047:39c2cab9e765], however some subsequent improvements were made and it is recommended to use [http://xenbits.xensource.com/xen-unstable.hg?rev/13690b68fd46 changeset: 18149:13690b68fd46] or newer.<br />
== Boot-Time VT-d Device Pass-Through ==<br />
As the name suggests, boot-time pass-through occurs at HVM domU (guest) boot-time. It is also referred to as ''static VT-d device pass-through'', however, this name is confusing as devices attached using this method may subsequently be hot-unplugged. To configure static pass-through for a domain add a "pci" line to the configuration file for the domain. For example, the following may be added to /etc/xen/hvm.conf:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0' ]<br />
</pre><br />
<br />
After updating the configuration file, start HVM guest and use "lspci" to verify that the the pass-through device is present in the guest. "ifconfig" or "ip addr show" may be used to see if IP address has been assigned to NIC devices.<br />
<br />
=== Virtual Slot Designation ===<br />
As of Xen 3.4.0 it is possible to specify the virtual slot that is used for by boot-time VT-d device pass-through. This is optional and when done is specified on a per-device basis by appending @SLOT_NUMBER to the BDF. For example, the following assigns 00:02.0 to virtual slot 7 while leaving xen to choose the slot for 01:00.0:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0@7' ]<br />
</pre><br />
<br />
== Viewing Devices That Have Been Passed-Through ==<br />
xm's pci-list command can be used to view the VT-d devices that have been passed-through to a domain. For example, below the devices attached to the HVM domain named HVMDomainVtd are shown.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list HVMDomainVtd<br />
VSlt domain bus slot func<br />
0x6 0x0 0x01 0x00 0x0<br />
0x7 0x0 0x00 0x02 0x0<br />
</pre><br />
<br />
Note that the domain field in the output above refers to the '''PCI domain not the Xen domain'''. PCI domains are a physical property of the host, as are PCI buses.<br />
<br />
== VT-d Device Hot-Plug ==<br />
VT-d device hot-plug involves attaching a device to an HVM domain once it has booted.<br />
<br />
xm's pci-attach command is used to perform hot-plug The device to be attached is specified using Self:BDFNotation and optionaly the desired virtual slot. If the virtual slot is omitted then a free one will be used. For example, the following command inserts the physical device into the HVM domain called HVMDomainVtd at virtual slot 7:<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7<br />
</pre><br />
<br />
As described above, xm pci-list can be used to verify the that the device is attached.<br />
<br />
== VT-d Device Hot-Unplug ==<br />
VT-d hot-unplug refers to detaching a pass-through device from a running HVM domain. The device may have been attached using boot-time VT-d device pass-through or VT-d device hot-plug.<br />
<br />
xm's pci-detach command is used to perform hot-unplug. The device to be detached is specified using Self:BDFNotation. The following command removes the device 0:2:0.0 from the domain named HVMDomainVtd.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0<br />
</pre><br />
<br />
As previously shown, pci-list can be used to show the devices that are attached to a domain and thus confirm that 0:2:0.0 has been detached as requested.<br />
<br />
== VT-d Device Hot-Plug/Unplug Usage Model ==<br />
* Live Migration: VT-d pass-through devices break live migration as physical device can't be save/restored. However, by hot-unplugging all the VT-d devices before live migration this problem is overcome.<br />
* Device Switching: VT-d hot-plug can be used to dynamically switch physical device between different HVM guest without shutdown.<br />
== Virtual Slots ==<br />
Up until Xen 3.4.0, slots 6 and 7 were reserved in HVM guests for VT-d pass-through. This meant that:<br />
<br />
* Only two pass-through devices could be attached<br />
* Attached devices would always be in slot 6 or 7<br />
* Slots 6 and 7 could not be used for IOEMU devices<br />
Xen 3.4.0 removes the reservation system. Any slot which isn't in use may be used for pass-through or IOEMU devices. Given that HVM domains have one virtual PCI bus, that a PCI bus has 32 slots for devices, and that a domain typically has a minimum of 3 IOEMU devices, 28 will be available for pass-through or extra IOEMU devices.<br />
<br />
== Enabling MSI/MSI-X for Assigned Devices ==<br />
* As of Xen 3.4.0 MSI/MSI-X is always on.<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/65dc37be0443 changeset: 18454:65dc37be0443]<br />
* In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1".<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/89d05940cc1c changeset: 18127:89d05940cc1c]<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi=1<br />
</pre><br />
<br />
* Prior to Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter" msi_irq_enable=1"<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi_irq_enable=1<br />
</pre><br />
<br />
== Single-Function and Multi-Function Pass-Through ==<br />
A PCI device may have up to 8 functions. A device with only one function is called a single-function device and a device with more than one function is called a multi-function device. A common example of a multi-function device are USB devices. For example, the following USB device is device 1a on bus 0 and has 3 functions: 0, 1 and 7.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# lspci -s 00:1a<br />
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)<br />
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)<br />
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)<br />
</pre><br />
<br />
As of Xen 3.4.1, and in all prior versions of Xen that supported VT-d pass-through, only single-function VT-d pass-through devices are supported in HVM domains. This means that if a function of as multi-function device is passed-through to an HVM domain, it will appear as function zero of a single-function device in the HVM domain, regardless of its physical function number.<br />
<br />
Single-function devices always have function zero present (and no other functions) and thus the function number does not change when a single-function device is passed through they are passed-through.<br />
<br />
The following table shows that regardless of the physical function number, the virtual function number will be zero.<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
| <br />
| Physical (dom0) <br />
|-<br />
| Single-Function <br />
| 00:02.0 <br />
|-<br />
| Multi-Function <br />
| 00:1a.1 <br />
|}<br />
<br />
Support for passing-through multi-function devices as multi-function devices has been merged into xen-unstable and should appear in 4.0. A back-port to 3.4.1 is available as per the [http://marc.info/?l=xen-devel&m=124748015304566&w=4 announcement email].<br />
<br />
This feature is accessed using extensions to BDF notation to allow a multi-function device to be specified as a single unit. For details please see Self:BDFNotation. Note that the back-port to 3.4.1 does not support the explicit mapping of virtual function numbers (the = syntax) as this is considered too green at this time.<br />
<br />
[[Category:Xen]]<br />
[[Category:HowTo]]<br />
[[Category:Users]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=VTd_HowTo&diff=5949VTd HowTo2013-01-18T18:05:29Z<p>Powerhouse: /* VT-d Enabled Systems */</p>
<hr />
<div><!-- MoinMoin name: VTdHowTo --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: VTdHowTo --><br />
<!-- Page revision: 00000104 --><br />
<!-- Original date: Sat Jul 30 03:13:48 2011 (1311995628000000) --><br />
<br />
{{TODO|This page is outdated. In particular the hardware sections need to be updated.}}<br />
<br />
__NOTOC__<br />
<!-- ## Please edit system and help pages ONLY in the moinmaster wiki! For more --><br />
<!-- ## information, please see [[MoinMaster]]:[[MoinPagesEditorGroup]]. --><br />
<!-- ##acl [[MoinPagesEditorGroup]]:read,write,delete,revert All:read --><br />
<!-- ##master-page:[[HelpTemplate]] --><br />
<!-- ##master-date:Unknown-Date --><br />
<!-- #format wiki --><br />
<!-- #language en --><br />
Please also check [[XenPCIpassthrough]] wiki page for more general information about Xen PCI passthru usage!<br />
<br />
VT-d Pass-Through is a technique to give a domU exclusive access to a PCI function using the IOMMU provided by VT-d. It is primarily targeted at HVM (fully virtualised) guests because PV (paravirtualized) pass-through does not require VT-d (altough it may be utilized too)<br />
<br />
== Xen 4.1 xl tools notes ==<br />
* Only devices with FLR capabilities are supported.<br />
* Passing through a PCI card without FLR capability will result an error.<br />
<br />
To check if your PCI devices have FLR function, check in this wiki, [[http://wiki.xen.org/wiki/Xen_PCI_Passthrough#How_can_I_check_if_PCI_device_supports_FLR_.28Function_Level_Reset.29_.3F How can I check if PCI device supports FLR (Function Level Reset) ?]]<br />
<br />
If you see output with "FLReset-" then your PCI device don't support FLR function. If output have "FLReset+" then it does.<br />
<br />
As this time of writing, June 2012, there are very very few PCI devices support FLR function.<br />
<br />
== Supported OSes ==<br />
* Dom0 (Host) OS: PAE, 64-bit<br />
* DomU (Guest) OS: 32-bit, PAE, 64-bit<br />
=== Tested DomO OSes ===<br />
* 64-bit host: 32/PAE/64 Linux/XP/Win2003/Vista guests<br />
* PAE host: 32/PAE Linux/XP/Win2003/Vista guests<br />
== VT-d Enabled Systems ==<br />
Note that in addition to the motherboard chipset and BIOS also your CPU must have support for IOMMU IO virtualization (VT-d)! Make sure to check your CPU vendors spec sheets for more information.<br />
<br />
CPUs known to work (with a motherboard from below list and a proper BIOS):<br />
<br />
(Intel reference the list of [http://ark.intel.com/search/advanced/?s=t&VTD=true compatible processors vt-d])<br />
<br />
* Intel [[Core2Duo]] (with VT-x)<br />
* Intel [[Core2Quad]]<br />
* Intel Core i7<br />
* Intel Core i5 (vPro Brand)<br />
* AMD FX-8120 / FX-8150<br />
For VT-d enabling work on Xen, we have been using development systems using following Intel motherboards:<br />
<br />
* DQ35MP<br />
* DQ35JO<br />
* Notes on VT-d compatibility:<br />
** VT-d is enabled on the following chipsets:<br />
*** Intel Q35 (desktop / workstation)<br />
*** Intel Q45 (desktop / workstation)<br />
*** Intel X58 (desktop / workstation)<br />
*** Intel X79<br />
*** Intel 55x0 (server)<br />
*** Intel 3450 (workstation / server)<br />
* The following chipsets have VT-d capability in theory, but most OEMs (such as Asus and Gigabyte) do not have it enabled on boards based on these:<br />
** Intel X38 (desktop / workstation)<br />
** Intel X48 (desktop / workstation)<br />
** Intel 32x0 (server)<br />
* For Intel Desktop Boards, these have VT-d support enabled:<br />
** Intel DQ35JO<br />
** Intel DQ35MP<br />
** Intel DX38BT, DX48BT2 (BIOS 1554 or 1782 required, VT-d is known to not work in other versions; iommu=workaround_bios_bug required) <br />
** Intel DQ45CB (BIOS 0061 required, previous versions are known to cause problems)<br />
** Intel DQ45EK<br />
** Intel DX58SO<br />
** Intel DQ67SW<br />
* For ASUS Desktop Boards, these have VT-d support enabled, but Asus does NOT support Linux, so you are on your own with any Linux or Xen issues like broken BIOSes:<br />
** ASUS P5E-VM DO (Intel Q35 chipset) requires IGD to be enabled (otherwise DMAR-table becomes corrupted)<br />
** ASUS P6T Deluxe (Intel X58 chipset) requires (currently non-public) BIOS update to correct DMAR-table issue<br />
** ASUS [[P6T6]] WS Revolution (Intel X58 chipset) incorrect bios DMAR-table<br />
** ASUS Sabertooth X58 (Intel X58 chipset) incorrect bios DMAR-table (2nd RMRR structure is incorrect)<br />
** ASUS Z8NA-D6 (dual processor nehalem board) works<br />
* Most server boards based on the Tylersburg chipset (55x0) and few boards based on 32x0 should have working VT-d, known examples are:<br />
** Intel server board: [http://www.intel.com/support/motherboards/server/S3200SH/sb/CS-028345.htm S3210SHLX] (BIOS >R0044 required.)<br />
** Supermicro server mainboard: [http://www.supermicro.com/products/motherboard/QPI/5500/X8DT3-F.cfm X8DT3-F]<br />
** Supermicro server 5026T-TB: [http://www.supermicro.com/products/system/2U/5026/SYS-5026T-T.cfm] Only work USB 1.1.<br />
* These motherboards are known to have broken BIOS preventing IO virtualization (VT-d IOMMU) from working:<br />
** Supermicro X7SB4 (with official BIOS 1.2a) has broken ACPI DMAR table with zero length entries. BIOS version 1.3 Beta fixes the problem.<br />
** Samsung X460 laptop: BIOS doesn't provide DMAR table so VT-d cannot be used.<br />
* As far as we know, following OEM systems also have VT-d enabled. Feel free to add others as they become available.<br />
** Dell: [http://www.dell.com/content/products/category.aspx/optix?c=us&cs=555&l=en&s=biz Optiplex 755], [http://www.dell.com/us/business/p/optiplex-980/pd Optiplex 980]<br />
** HP Compaq: [http://h10010.www1.hp.com/wwpc/us/en/en/WF04a/12454-12454-64287-321860-3328898.html DC7800]<br />
** Fujitsu-Siemens: [http://facts.fujitsu-siemens.com/index.cfm?fuseaction=dspcontent&lid=134&page=3 Esprimo 5925]<br />
** Lenovo: [http://shop.lenovo.com/us/workstations/thinkstation/workstation/e20 ThinkStation E20] and Thinkserver TS200v<br />
** HP: [http://www.hp.com/united-states/campaigns/workstations/z420_features.html#.UG6055gxrug Z420 Workstation] <br />
*VT-d Compatible ASRock Motherboards (Socket 1155 and 2011) probably all Z68 and Z77 board's<br />
**ASRock H77 Pro4-M<br />
**ASRock Z68 Extreme4 Gen3<br />
**ASRock Z68 Professional Gen3<br />
**ASRock Z77 Pro3<br />
**ASRock Z77 Extreme4 up to BIOS 2.50 ('''starting from BIOS release 2.70 there is no VT-d support !!!'''), see [http://www.asrock.com/mb/Intel/Z77%20Extreme4/index.us.asp?cat=Download&os=BIOS] and [http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine/480#post_19078253]<br />
**ASRock Z77 Extreme6 up to BIOS 2.30 ('''starting from BIOS release 2.40 there is no VT-d support !!!'''), see [http://www.asrock.com/mb/Intel/Z77%20Extreme6/index.us.asp?cat=Download&os=BIOS] and [http://www.overclock.net/t/1205216/guide-create-a-gaming-virtual-machine/480#post_19078253]<br />
**ASRock Z77 Professional<br />
**ASRock X79 Extreme9 (socket 2011)<br />
* Gigabyte Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** GA-X79-UD5 (rev. 1.0)<br />
** GA-X79-UD7 (rev. 1.0)<br />
** GA-Z77X-D3H (rev. 1.0)<br />
* MSI 1155 Motherboards (Socket 1155) suggested by manufacturer on inquiry<br />
** Z68A-GD80-G3<br />
** Z77A-GD80<br />
** Z77A-GD65<br />
** Z77A-GD55<br />
** Z77A-G45<br />
** Z77A-G43<br />
<br />
== AMD desktop chipsets with IOMMU support ==<br />
* AMD 890FX chipset supports IOMMU. Other 890 chipsets don't have IOMMU support!<br />
* AMD 990FX, 990X and 970 chipsets support IOMMU.<br />
* Even when the chipset supports IOMMU, the bios must have a ACPI IVRS table to enable the use of it! So actual support depends on the motherboard manufacturer. At the time of writing all motherboards seem to have a (beta)bios available supporting the IOMMU. A thread with user expiriences can also be found at this forum: http://forums.tweaktown.com/f69/ga-890fxa-ud5-iommu-bios-switch-39801/<br />
* Motherboards with a BIOS supporting the IOMMU(as reported by users):<br />
** ASUS Crosshair IV (reported working by Jens Krehbiel-Gräther)<br />
** ASUS Crosshair V Formula (reported working by Pavel Matěja)<br />
** ASUS F2A85-V PRO<br />
** ASUS M4A89TD Pro/USB3 (reported working by Jens Krehbiel-Gräther)<br />
** Asrock 890FX Deluxe3 (reported working by Jens Krehbiel-Gräther)<br />
** Biostar TA890FXE (from bios version 89FAD629.BST reported working by Joop Boonen, Konrad Rzeszutek Wilk)<br />
* Motherboards with a beta-bios available from tech-support that supports the IOMMU:<br />
** Gigabyte GA-890FXA-UD5<br />
** Gigabyte GA-890FXA-UD7<br />
** MSI 890FXA-GD70 (from beta-bios 1.75 reported working by Sander Eikelenboom)<br />
<br />
== AMD server (opteron) chipsets with IOMMU support ==<br />
* AMD SR5690 / SR5670 (Tyan S8212)<br />
== Caveat on Conventional PCI Device Pass-Through ==<br />
* The VT-d specification states that all conventional PCI devices behind a PCIe-to-PCI bridge have to be assigned to the same domain.<br />
* PCIe devices do not have this restriction.<br />
== Notes on Compiling Source Code ==<br />
It is not the intention of this document to describe in detail how to compile and install Xen from source. Rather, the intention of this section is to highlight points of particular note when compiling from source for use with VT-d.<br />
<br />
=== Dom0 Kernel ===<br />
A prerequisite for binding devices to pciback, which is discussed in more detail below, is that pciback is compiled statically into or as a module for the dom0 kernel. This means that in the kernel config CONFIG_XEN_PCIDEV_BACKEND=y should be set to statically compile pciback into the kernel or CONFIG_XEN_PCIDEV_BACKEND=m should be set to compile pciback as a module.<br />
<br />
For pv-ops dom0, you can use pci-stub to hide device. Set CONFIG_PCI_STUB=y to build it into kernel or CONFIG_PCI_STUB=m to build as a module.<br />
<br />
=== Device Model ===<br />
When building the device model, also known as qemu-dm and qemu-xen, often automatically downloaded during compilation and found in the tools/ioemu-dir/ subdirectory of the xen tree, it is important that the development headers and libraries for libpci are installed. Otherwise qemu-dm will be built without pass-through support.<br />
<br />
If the device model has been compiled without pass-through support, the following error will show up in xend.log at run-time when pass-through is attempted.<br />
<br />
* <br />
<pre><br />
ERROR (XendDomainInfo:581) Device model didn't tell the vslots for PCI device<br />
</pre><br />
<br />
== VT-d boot parameter: iommu ==<br />
VT-d is disabled by default, to enable it, need 'iommu' parameter to enable it.<br />
<br />
* off|no|false|disable: Disable IOMMU (default)<br />
* pv: Enable IOMMU for PV domains<br />
* no-pv: Disable IOMMU for PV domains (default)<br />
* force|required: Don't boot unless IOMMU is enabled<br />
* workaround_bios_bug: Workaround some bios issues to still enable VT-d, don't guarantee security<br />
* pass-through: Enable VT-d DMA pass-through (no DMA translation for Dom0)<br />
* no-snoop: Disable VT-d Snoop Control<br />
* no-qinval: Disable VT-d Queued Invalidation<br />
* no-intremap: Disable VT-d Interrupt Remapping<br />
* verbose: In Xen 4.0.0 and newer, enable verbose logging while enabling IOMMU and parsing ACPI DMAR tables<br />
Usually, you just need 'iommu=1' to enable VT-d. At the same time, most of VT-d features (DMA remapping, snoop control, queued invalidation and interrupt remapping) are enabled by default if they are available. You can use 'no-xxx' to disable a feature, for example, 'iommu=no-snoop' disable snoop control.<br />
<br />
When RMRR address range is not in reserved memory (BIOS issue), can use 'iommu_inclusive_mapping=1' to work around it.<br />
<br />
The grub configuration is like:<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
== Binding Devices to pciback ==<br />
In order to pass-through devices using VT-d then need to be bound to pciback to ensure that they are not bound to another dom0 driver and this free for use by pass-through. It is possible to view this as hiding the device from Dom0.<br />
<br />
=== Binding at Boot-Time with "old-style" Xen 2.6.18 dom0 Linux kernel ===<br />
If pciback is statically compiled into the kernel, then perhaps the simplest to make PCI devices available for VT-d pass-through is to use the pciback.hide kernel parameter, as illustrated in the following grub configuration snippet. .<br />
<br />
<br />
<pre><br />
title Xen-Linux (2.6.18-xen)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.18.8-xen root=LABEL=/ ro pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.18-xen.img<br />
</pre><br />
<br />
The arguments to pciback.hide are PCI functions in Self:BDFNotation, enclosed in brackets. The output of the lspci command may be useful when selecting PCI functions to hide.<br />
<br />
=== Binding at Boot-Time with pvops dom0 Linux kernel (2.6.31, 2.6.32, and newer) ===<br />
<br />
<pre><br />
title Xen-Linux (2.6.32.9)<br />
root (hd0,0)<br />
kernel /boot/xen.gz iommu=1<br />
module /boot/vmlinuz-2.6.32.9 root=LABEL=/ ro xen-pciback.hide=(01:00.0)(00:02.0)<br />
module /boot/initrd-2.6.32.9.img<br />
</pre><br />
<br />
The pciback driver name was changed from "pciback" to "xen-pciback" in December 2009 in pvops kernels.<br />
<br />
=== Binding at Run-Time ===<br />
As discussed in [[Assign_hardware_to_DomU_with_PCIBack_as_module]], it is possible to bind a device to pciback after dom0 has booted, even if it has been bound to another device in the mean time. This method can be used both when pciback is statically compiled into the dom0 kernel and when it has been compiled as a module.<br />
<br />
=== Binding at Module Insertion Time ===<br />
As also discussed in Self:Assign_hardware_to_DomU_with_PCIBack_as_module, it is possible to bind a device to pciback at the time that the pciback module is inserted. This technique is only useful if pciback has been compiled as a module.<br />
<br />
== Binding Devices to pci-stub ==<br />
If using pv-ops dom0, also can use pci-stub to hide devices for assignment (example PCI device 01:00.0) * lspci -n * locate the entry for device 01:00.0 and note down the vendor & device ID 8086:10b9 .<br />
<br />
Note: Binding devices with pci-stub only works for pci passthrough to HVM guests! Hiding with xen-pciback as mentioned above works for both HVM and PV guests and is preferred.<br />
<br />
<br />
<pre><br />
...<br />
01:00.0 0200: 8086:10b9 (rev 06)<br />
...<br />
</pre><br />
<br />
* then use following commands to hide it: .<br />
<br />
<br />
<pre><br />
echo "8086 10b9" > /sys/bus/pci/drivers/pci-stub/new_id<br />
echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind<br />
echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind<br />
</pre><br />
<br />
== Viewing Devices that are Available for Pass-Through ==<br />
xm's pci-list-assignable-devices can be used to list devices that are available for pass-through. That is, devices that have been hidden from dom0 by pciback.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list-assignable-devices<br />
0000:01:00.0<br />
0000:00:02.0<br />
</pre><br />
<br />
The devices are shown in Extended BDF notation, for more information see Self:BDFNotation.<br />
<br />
=== Implementation Details ===<br />
xm finds all devices the devices owned by pciback, checks if they have proper FLR method, checks if they have page-aligned MMIO BARs, and checks if the devices have been already assigned, finally it prints out the assignable devices. xm assumes the integrated devices (whose bus number is 0) have a proper FLR method.<br />
<br />
=== Availability of this Feature ===<br />
* Release History: Available from Xen 3.3.0<br />
* Development History: Available from [http://xenbits.xensource.com/xen-unstable.hg?rev/39c2cab9e765 changeset: 18047:39c2cab9e765], however some subsequent improvements were made and it is recommended to use [http://xenbits.xensource.com/xen-unstable.hg?rev/13690b68fd46 changeset: 18149:13690b68fd46] or newer.<br />
== Boot-Time VT-d Device Pass-Through ==<br />
As the name suggests, boot-time pass-through occurs at HVM domU (guest) boot-time. It is also referred to as ''static VT-d device pass-through'', however, this name is confusing as devices attached using this method may subsequently be hot-unplugged. To configure static pass-through for a domain add a "pci" line to the configuration file for the domain. For example, the following may be added to /etc/xen/hvm.conf:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0' ]<br />
</pre><br />
<br />
After updating the configuration file, start HVM guest and use "lspci" to verify that the the pass-through device is present in the guest. "ifconfig" or "ip addr show" may be used to see if IP address has been assigned to NIC devices.<br />
<br />
=== Virtual Slot Designation ===<br />
As of Xen 3.4.0 it is possible to specify the virtual slot that is used for by boot-time VT-d device pass-through. This is optional and when done is specified on a per-device basis by appending @SLOT_NUMBER to the BDF. For example, the following assigns 00:02.0 to virtual slot 7 while leaving xen to choose the slot for 01:00.0:<br />
<br />
* <br />
<pre><br />
pci = [ '01:00.0', '00:02.0@7' ]<br />
</pre><br />
<br />
== Viewing Devices That Have Been Passed-Through ==<br />
xm's pci-list command can be used to view the VT-d devices that have been passed-through to a domain. For example, below the devices attached to the HVM domain named HVMDomainVtd are shown.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-list HVMDomainVtd<br />
VSlt domain bus slot func<br />
0x6 0x0 0x01 0x00 0x0<br />
0x7 0x0 0x00 0x02 0x0<br />
</pre><br />
<br />
Note that the domain field in the output above refers to the '''PCI domain not the Xen domain'''. PCI domains are a physical property of the host, as are PCI buses.<br />
<br />
== VT-d Device Hot-Plug ==<br />
VT-d device hot-plug involves attaching a device to an HVM domain once it has booted.<br />
<br />
xm's pci-attach command is used to perform hot-plug The device to be attached is specified using Self:BDFNotation and optionaly the desired virtual slot. If the virtual slot is omitted then a free one will be used. For example, the following command inserts the physical device into the HVM domain called HVMDomainVtd at virtual slot 7:<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-attach HVMDomainVtd 0:2:0.0 7<br />
</pre><br />
<br />
As described above, xm pci-list can be used to verify the that the device is attached.<br />
<br />
== VT-d Device Hot-Unplug ==<br />
VT-d hot-unplug refers to detaching a pass-through device from a running HVM domain. The device may have been attached using boot-time VT-d device pass-through or VT-d device hot-plug.<br />
<br />
xm's pci-detach command is used to perform hot-unplug. The device to be detached is specified using Self:BDFNotation. The following command removes the device 0:2:0.0 from the domain named HVMDomainVtd.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# xm pci-detach HVMDomainVtd 0:2:0.0<br />
</pre><br />
<br />
As previously shown, pci-list can be used to show the devices that are attached to a domain and thus confirm that 0:2:0.0 has been detached as requested.<br />
<br />
== VT-d Device Hot-Plug/Unplug Usage Model ==<br />
* Live Migration: VT-d pass-through devices break live migration as physical device can't be save/restored. However, by hot-unplugging all the VT-d devices before live migration this problem is overcome.<br />
* Device Switching: VT-d hot-plug can be used to dynamically switch physical device between different HVM guest without shutdown.<br />
== Virtual Slots ==<br />
Up until Xen 3.4.0, slots 6 and 7 were reserved in HVM guests for VT-d pass-through. This meant that:<br />
<br />
* Only two pass-through devices could be attached<br />
* Attached devices would always be in slot 6 or 7<br />
* Slots 6 and 7 could not be used for IOEMU devices<br />
Xen 3.4.0 removes the reservation system. Any slot which isn't in use may be used for pass-through or IOEMU devices. Given that HVM domains have one virtual PCI bus, that a PCI bus has 32 slots for devices, and that a domain typically has a minimum of 3 IOEMU devices, 28 will be available for pass-through or extra IOEMU devices.<br />
<br />
== Enabling MSI/MSI-X for Assigned Devices ==<br />
* As of Xen 3.4.0 MSI/MSI-X is always on.<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/65dc37be0443 changeset: 18454:65dc37be0443]<br />
* In Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter "msi=1".<br />
** This change was made in [http://xenbits.xensource.com/xen-unstable.hg?rev/89d05940cc1c changeset: 18127:89d05940cc1c]<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi=1<br />
</pre><br />
<br />
* Prior to Xen 3.3.0, MSI/MSI-X could be enabled using the xen boot parameter" msi_irq_enable=1"<br />
** The following snippet from a grub configuration enables MSI/MSI-X<br />
<br />
<pre><br />
kernel xen.gz msi_irq_enable=1<br />
</pre><br />
<br />
== Single-Function and Multi-Function Pass-Through ==<br />
A PCI device may have up to 8 functions. A device with only one function is called a single-function device and a device with more than one function is called a multi-function device. A common example of a multi-function device are USB devices. For example, the following USB device is device 1a on bus 0 and has 3 functions: 0, 1 and 7.<br />
<br />
* <br />
<pre><br />
[root@vt-vtd ~]# lspci -s 00:1a<br />
00:1a.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)<br />
00:1a.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)<br />
00:1a.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)<br />
</pre><br />
<br />
As of Xen 3.4.1, and in all prior versions of Xen that supported VT-d pass-through, only single-function VT-d pass-through devices are supported in HVM domains. This means that if a function of as multi-function device is passed-through to an HVM domain, it will appear as function zero of a single-function device in the HVM domain, regardless of its physical function number.<br />
<br />
Single-function devices always have function zero present (and no other functions) and thus the function number does not change when a single-function device is passed through they are passed-through.<br />
<br />
The following table shows that regardless of the physical function number, the virtual function number will be zero.<br />
{| border="1" cellpadding="2" cellspacing="0"<br />
| <br />
| Physical (dom0) <br />
|-<br />
| Single-Function <br />
| 00:02.0 <br />
|-<br />
| Multi-Function <br />
| 00:1a.1 <br />
|}<br />
<br />
Support for passing-through multi-function devices as multi-function devices has been merged into xen-unstable and should appear in 4.0. A back-port to 3.4.1 is available as per the [http://marc.info/?l=xen-devel&m=124748015304566&w=4 announcement email].<br />
<br />
This feature is accessed using extensions to BDF notation to allow a multi-function device to be specified as a single unit. For details please see Self:BDFNotation. Note that the back-port to 3.4.1 does not support the explicit mapping of virtual function numbers (the = syntax) as this is considered too green at this time.<br />
<br />
[[Category:Xen]]<br />
[[Category:HowTo]]<br />
[[Category:Users]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=Bind_lib.bash_:_A_script_that_makes_pciback_with_modules_easier_to_use&diff=5822Bind lib.bash : A script that makes pciback with modules easier to use2012-12-15T21:16:57Z<p>Powerhouse: </p>
<hr />
<div>; Description<br />
<br />
A script that makes pciback with modules easier than with recompiling the kernel and using the boot loader line.<br />
<br />
It is based on the info from [http://old-list-archives.xen.org/archives/html/xen-devel/2010-03/msg00448.html this email].<br />
<br />
{{Info|In Xen 4.2 with the xl toolstack this functionality is [[Xen 4.2: xl and pci pass-through|integrated into the toolstack]]}}<br />
<br />
; Instructions<br />
<br />
This script should be run like this:<br />
<br />
<code><br />
. bind_lib.bash<br />
</code><br />
<br />
And then call the functions yourself, for example:<br />
<br />
<code><br />
modprobe xen-pciback<br />
bindback "0000:04:00.0"<br />
</code><br />
<br />
This script is tested with a Radeon 6770 card, using pciback, with "pci_passthru=0" and "stdvga=1". Using this method, you don't need to blacklist radeon. <br />
<br />
<br />
<br />
The functions:<br />
<br />
; bind_lib.bash<br />
<br />
<code><br />
#!/bin/bash<br />
#<br />
# License: GPLv2<br />
# Author: Peter Maloney<br />
#<br />
# Script to bind devices to pciback (or pci-stub)<br />
<br />
find_new_id() {<br />
device="$1"<br />
len=${#device}<br />
if [ "$len" -eq 12 ]; then<br />
device="${device:5:12}"<br />
fi<br />
lspci -n | grep "${device}" | cut -d' ' -f3 | sed -r "s/:/ /"<br />
}<br />
<br />
bindstub() {<br />
device="$1"<br />
echo "binddevice $device"<br />
<br />
if [ ! -e "/sys/bus/pci/devices/$device" ]; then<br />
echo " ERROR: Device does not exist... cancelling"<br />
return<br />
fi<br />
<br />
# with pci-stub, you do new_id, then unbind, then bind<br />
<br />
echo "create new_id"<br />
chmod +w /sys/bus/pci/drivers/pci-stub/new_id<br />
new_id="$(find_new_id "$device")"<br />
echo " echo \"$new_id\" > /sys/bus/pci/drivers/pci-stub/new_id"<br />
echo "$new_id" > /sys/bus/pci/drivers/pci-stub/new_id<br />
<br />
echo "unbind"<br />
if [ ! -e "/sys/bus/pci/devices/$device/driver" ]; then<br />
echo " no driver to unbind"<br />
else<br />
chmod +w "/sys/bus/pci/devices/${device}/driver/unbind"<br />
echo " echo -n \"$device\" > \"/sys/bus/pci/devices/$device/driver/unbind\""<br />
echo -n "$device" > "/sys/bus/pci/devices/$device/driver/unbind"<br />
fi<br />
<br />
echo "bind"<br />
chmod +w /sys/bus/pci/drivers/pci-stub/bind<br />
echo " echo -n \"$device\" > /sys/bus/pci/drivers/pci-stub/bind"<br />
echo -n "$device" > /sys/bus/pci/drivers/pci-stub/bind<br />
<br />
echo<br />
}<br />
<br />
bindback() {<br />
device="$1"<br />
echo "binddevice $device"<br />
<br />
if [ ! -e "/sys/bus/pci/devices/$device" ]; then<br />
echo "ERROR: Device does not exist... cancelling"<br />
return<br />
fi<br />
<br />
# with pciback, you do unbind, then new_slot, then bind<br />
<br />
echo "unbind"<br />
if [ ! -e "/sys/bus/pci/devices/$device/driver" ]; then<br />
echo " no driver to unbind"<br />
else<br />
chmod +w "/sys/bus/pci/devices/${device}/driver/unbind"<br />
echo " echo -n \"$device\" > \"/sys/bus/pci/devices/$device/driver/unbind\""<br />
echo -n "$device" > "/sys/bus/pci/devices/$device/driver/unbind"<br />
fi<br />
<br />
echo "create new_slot"<br />
chmod +w /sys/bus/pci/drivers/pciback/new_slot<br />
echo " echo -n \"$device\" > /sys/bus/pci/drivers/pciback/new_slot"<br />
echo -n "$device" > /sys/bus/pci/drivers/pciback/new_slot<br />
<br />
echo "bind"<br />
chmod +w /sys/bus/pci/drivers/pciback/bind<br />
echo " echo -n \"$device\" > /sys/bus/pci/drivers/pciback/bind"<br />
echo -n "$device" > /sys/bus/pci/drivers/pciback/bind<br />
<br />
echo<br />
}<br />
<br />
</code><br />
<br />
<br />
Examples for binding devices (with hardcoded PCI address examples):<br />
<br />
<code><br />
modprobe pci-stub<br />
# vga2<br />
bindstub "0000:04:00.0"<br />
<br />
# vga2 audio<br />
bindstub "0000:04:00.1"<br />
<br />
# usb2<br />
bindstub "0000:00:12.0"<br />
bindstub "0000:00:12.2"<br />
<br />
# usb3<br />
bindstub "0000:02:00.0"<br />
<br />
<br />
# Check results<br />
ls -l /sys/bus/pci/devices/{0000:01:00.0,0000:01:00.1,0000:04:00.0,0000:04:00.1,0000:00:12.0,0000:00:12.2,0000:02:00.0}/driver<br />
xm pci-list-assignable-devices<br />
</code><br />
<br />
<code><br />
modprobe xen-pciback<br />
# vga2<br />
bindback "0000:04:00.0"<br />
<br />
# vga2 audio<br />
bindback "0000:04:00.1"<br />
<br />
# usb2<br />
bindback "0000:00:12.0"<br />
bindback "0000:00:12.2"<br />
<br />
# usb3<br />
bindback "0000:02:00.0"<br />
<br />
# Check results<br />
ls -l /sys/bus/pci/devices/{0000:01:00.0,0000:01:00.1,0000:04:00.0,0000:04:00.1,0000:00:12.0,0000:00:12.2,0000:02:00.0}/driver<br />
xm pci-list-assignable-devices<br />
</code><br />
<br />
[[Category:User]]<br />
[[Category:Xen]]<br />
[[Category:VGA]]<br />
[[Category:Example]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=Xen_VGA_Passthrough_Tested_Adapters&diff=5821Xen VGA Passthrough Tested Adapters2012-12-15T20:05:03Z<p>Powerhouse: /* Nvidia display adapters */</p>
<hr />
<div><!-- MoinMoin name: XenVGAPassthroughTestedAdapters --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: XenVGAPassthroughTestedAdapters --><br />
<!-- Page revision: 00000019 --><br />
<!-- Original date: Wed Jul 20 19:45:15 2011 (1311191115000000) --><br />
<br />
{{Info|We only recently started collecting this information and rely on you to let us know how it works for you!}}<br />
<br />
= List of tested graphics adapters for Xen VGA Passthrough =<br />
Please send an email to xen-devel or xen-users with all the details about your VGA graphics passthru experience. '''If you are confident that your adaptor works, feel free to just add it to this page.''' <br />
<br />
Please include the following details in the email:<br />
<br />
* Xen version<br />
* Dom0 kernel version, and if it's pvops or xenlinux based<br />
* "lspci" output for the graphics card to get the PCI IDs and model information<br />
* Did you passthru all the PCI IDs or just one?<br />
* Was it primary or secondary graphics adapter?<br />
* Motherboard vendor/model and BIOS version.<br />
* Guest OS and driver version.<br />
* How did it work.. did you have some issues?<br />
Xen VGA graphics passthru has been tested to work with these graphics adapters:<br />
<br />
== Intel display adapters ==<br />
<br />
<pre><br />
Xen 4.0.0, OpenSuse 11.2 xenlinux 2.6.31.12-0.2 dom0 kernel. Dell Optiplex 755 desktop. <br />
Windows XP HVM guest, Dell provided Intel graphics driver. Primary adapter. Passthru for both PCI IDs. <br />
<br />
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02) <br />
00:02.1 Display controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)<br />
</pre><br />
<br />
== Nvidia display adapters ==<br />
Note that Nvidia officially supports only Quadro FX 3800, 4800 and 5800 for graphics passthrough usage (they've tested and verified their binary drivers for these graphics cards in combination with graphics passthrough).<br />
<br />
In addition, Nvidia lists the following graphics adapters as "Multi-OS" capable: Quadro 6000, 5000, 4000, 2000. "Multi-OS" allows VGA passthrough to fully virtualized guests. Note: The Nvidia Quadro 600 is '''not''' supported.<br />
<br />
User reported working adapters:<br />
<br />
* Nvidia Quadro 2000, as secondary display adapter using Linux Mint 13 with Xen 4.1.2 hypervisor and Nvidia driver release 305.93 for Windows 7<br />
<br />
<pre><br />
Xen 4.0.0-rc8, pvops xen/stable-2.6.32.x dom0 kernel. Asus P7P55D MB, BIOS 1207, Core i7 860 CPU. <br />
Linux HVM guest with nv driver. Primary adapter. Passthru for single/only PCI ID. hidden from dom0: xen-pciback.hide=(01:00.0) <br />
<br />
01:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 8800 GTS 512] (rev a2)<br />
</pre><br />
<br />
== ATI/AMD display adapters ==<br />
AMD sent patch enabling VGA passthrough on AMD/ATI Radeon/FirePro graphics adapters on 2010-10-07: http://lists.xensource.com/archives/html/xen-devel/2010-10/msg00284.html .<br />
<br />
Tested adapters by AMD: <br />
<br />
* Radeon 4850<br />
* Radeon 4870<br />
* [[FirePro]] V5700<br />
* [[FirePro]] M5800<br />
<br />
User reported working adapters: <br />
<br />
* ATI HD4550<br />
* Radeon 4770 on Fedora 12, WinXP and Win7<br />
* ATI HD5750, using the Catalyst 10-9 driver on Win7 32bit<br />
* ATI HD5850<br />
* ATI HD6770 (tested with pciback under WinXP, xen 4.1.2 and 4.1.3)<br />
* ATI HD6850 under WinXP and Win7<br />
* ATI Radeon HD 6970 under Win7 64bit<br />
* AMD HD7970 under Win7<br />
* AMD HD7950 under Win7 64bit (Catalyst 12.10), Arch Linux 64bit (Catalyst 12.11) using XEN 4.2 (pciback)<br />
* AMD HD6870 under Win7 64bit using Xen 4.2(only with xl tools) <br />
<br />
There are certain user-reports that the Catalyst Control Center (ccc.exe) should be _not_ installed when installing the Catalyst Driver - for unknown reasons it may cause a BSOD. The GPU Driver itself works without problems when not installing CCC.<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel WSM-EP, Integrated graphics. <br />
RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0b:01.0). <br />
<br />
0b:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)<br />
</pre><br />
<br />
Xen 4.2 unstable changeset 25167 (Xen 4.1.2, 4.1.3, and many 4.2 changesets as well), Debian Wheezy, EFI grub, Kernel 3.3.2 (and 3.3.0), ATI Radeon HD 6870, with some difficulty installing the Catalyst drivers.<br />
<br />
== Matrox display adapters ==<br />
Tested adapters: MGA G200e<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel Romley-EP, Integrated graphics. <br />
Windows8 Beta HVM guest, RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0a:00.0). <br />
<br />
0a:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200e [Pilot] ServerEngines (SEP1) (rev 04)<br />
</pre><br />
<br />
<br />
== Links ==<br />
See [[XenVGAPassthrough]] wiki page for more information about Xen VGA graphics passthru in general.<br />
<br />
[[Category:Xen]]<br />
[[Category:Users]]<br />
[[Category:Developers]]<br />
[[Category:Compatibility]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=Xen_VGA_Passthrough_Tested_Adapters&diff=5820Xen VGA Passthrough Tested Adapters2012-12-15T19:49:14Z<p>Powerhouse: /* Nvidia display adapters */</p>
<hr />
<div><!-- MoinMoin name: XenVGAPassthroughTestedAdapters --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: XenVGAPassthroughTestedAdapters --><br />
<!-- Page revision: 00000019 --><br />
<!-- Original date: Wed Jul 20 19:45:15 2011 (1311191115000000) --><br />
<br />
{{Info|We only recently started collecting this information and rely on you to let us know how it works for you!}}<br />
<br />
= List of tested graphics adapters for Xen VGA Passthrough =<br />
Please send an email to xen-devel or xen-users with all the details about your VGA graphics passthru experience. '''If you are confident that your adaptor works, feel free to just add it to this page.''' <br />
<br />
Please include the following details in the email:<br />
<br />
* Xen version<br />
* Dom0 kernel version, and if it's pvops or xenlinux based<br />
* "lspci" output for the graphics card to get the PCI IDs and model information<br />
* Did you passthru all the PCI IDs or just one?<br />
* Was it primary or secondary graphics adapter?<br />
* Motherboard vendor/model and BIOS version.<br />
* Guest OS and driver version.<br />
* How did it work.. did you have some issues?<br />
Xen VGA graphics passthru has been tested to work with these graphics adapters:<br />
<br />
== Intel display adapters ==<br />
<br />
<pre><br />
Xen 4.0.0, OpenSuse 11.2 xenlinux 2.6.31.12-0.2 dom0 kernel. Dell Optiplex 755 desktop. <br />
Windows XP HVM guest, Dell provided Intel graphics driver. Primary adapter. Passthru for both PCI IDs. <br />
<br />
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02) <br />
00:02.1 Display controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)<br />
</pre><br />
<br />
== Nvidia display adapters ==<br />
Note that Nvidia officially supports only Quadro FX 3800, 4800 and 5800 for graphics passthrough usage (they've tested and verified their binary drivers for these graphics cards in combination with graphics passthrough).<br />
<br />
The Nvidia Quadro 2000 graphics adapter has been reported to work with VGA passthrough as secondary display adapter, using a Xen 4.1.2 hypervisor. In addition, Nvidia lists the following graphics adapters as "Multi-OS" capable: Quadro 6000, 5000, 4000, 2000. "Multi-OS" allows VGA passthrough to fully virtualized guests.<br />
<br />
Nvidia Quadro 600 (and 450) are '''not''' supported.<br />
<br />
<br />
<pre><br />
Xen 4.0.0-rc8, pvops xen/stable-2.6.32.x dom0 kernel. Asus P7P55D MB, BIOS 1207, Core i7 860 CPU. <br />
Linux HVM guest with nv driver. Primary adapter. Passthru for single/only PCI ID. hidden from dom0: xen-pciback.hide=(01:00.0) <br />
<br />
01:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 8800 GTS 512] (rev a2)<br />
</pre><br />
<br />
== ATI/AMD display adapters ==<br />
AMD sent patch enabling VGA passthrough on AMD/ATI Radeon/FirePro graphics adapters on 2010-10-07: http://lists.xensource.com/archives/html/xen-devel/2010-10/msg00284.html .<br />
<br />
Tested adapters by AMD: <br />
<br />
* Radeon 4850<br />
* Radeon 4870<br />
* [[FirePro]] V5700<br />
* [[FirePro]] M5800<br />
<br />
User reported working adapters: <br />
<br />
* ATI HD4550<br />
* Radeon 4770 on Fedora 12, WinXP and Win7<br />
* ATI HD5750, using the Catalyst 10-9 driver on Win7 32bit<br />
* ATI HD5850<br />
* ATI HD6770 (tested with pciback under WinXP, xen 4.1.2 and 4.1.3)<br />
* ATI HD6850 under WinXP and Win7<br />
* ATI Radeon HD 6970 under Win7 64bit<br />
* AMD HD7970 under Win7<br />
* AMD HD7950 under Win7 64bit (Catalyst 12.10), Arch Linux 64bit (Catalyst 12.11) using XEN 4.2 (pciback)<br />
* AMD HD6870 under Win7 64bit using Xen 4.2(only with xl tools) <br />
<br />
There are certain user-reports that the Catalyst Control Center (ccc.exe) should be _not_ installed when installing the Catalyst Driver - for unknown reasons it may cause a BSOD. The GPU Driver itself works without problems when not installing CCC.<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel WSM-EP, Integrated graphics. <br />
RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0b:01.0). <br />
<br />
0b:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)<br />
</pre><br />
<br />
Xen 4.2 unstable changeset 25167 (Xen 4.1.2, 4.1.3, and many 4.2 changesets as well), Debian Wheezy, EFI grub, Kernel 3.3.2 (and 3.3.0), ATI Radeon HD 6870, with some difficulty installing the Catalyst drivers.<br />
<br />
== Matrox display adapters ==<br />
Tested adapters: MGA G200e<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel Romley-EP, Integrated graphics. <br />
Windows8 Beta HVM guest, RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0a:00.0). <br />
<br />
0a:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200e [Pilot] ServerEngines (SEP1) (rev 04)<br />
</pre><br />
<br />
<br />
== Links ==<br />
See [[XenVGAPassthrough]] wiki page for more information about Xen VGA graphics passthru in general.<br />
<br />
[[Category:Xen]]<br />
[[Category:Users]]<br />
[[Category:Developers]]<br />
[[Category:Compatibility]]<br />
[[Category:VGA]]</div>Powerhousehttps://wiki.xenproject.org/index.php?title=Xen_VGA_Passthrough_Tested_Adapters&diff=5819Xen VGA Passthrough Tested Adapters2012-12-15T19:03:03Z<p>Powerhouse: /* Nvidia display adapters */</p>
<hr />
<div><!-- MoinMoin name: XenVGAPassthroughTestedAdapters --><br />
<!-- Comment: --><br />
<!-- WikiMedia name: XenVGAPassthroughTestedAdapters --><br />
<!-- Page revision: 00000019 --><br />
<!-- Original date: Wed Jul 20 19:45:15 2011 (1311191115000000) --><br />
<br />
{{Info|We only recently started collecting this information and rely on you to let us know how it works for you!}}<br />
<br />
= List of tested graphics adapters for Xen VGA Passthrough =<br />
Please send an email to xen-devel or xen-users with all the details about your VGA graphics passthru experience. '''If you are confident that your adaptor works, feel free to just add it to this page.''' <br />
<br />
Please include the following details in the email:<br />
<br />
* Xen version<br />
* Dom0 kernel version, and if it's pvops or xenlinux based<br />
* "lspci" output for the graphics card to get the PCI IDs and model information<br />
* Did you passthru all the PCI IDs or just one?<br />
* Was it primary or secondary graphics adapter?<br />
* Motherboard vendor/model and BIOS version.<br />
* Guest OS and driver version.<br />
* How did it work.. did you have some issues?<br />
Xen VGA graphics passthru has been tested to work with these graphics adapters:<br />
<br />
== Intel display adapters ==<br />
<br />
<pre><br />
Xen 4.0.0, OpenSuse 11.2 xenlinux 2.6.31.12-0.2 dom0 kernel. Dell Optiplex 755 desktop. <br />
Windows XP HVM guest, Dell provided Intel graphics driver. Primary adapter. Passthru for both PCI IDs. <br />
<br />
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02) <br />
00:02.1 Display controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)<br />
</pre><br />
<br />
== Nvidia display adapters ==<br />
Note that Nvidia officially supports only Quadro FX 3800, 4800 and 5800 for graphics passthrough usage (they've tested and verified their binary drivers for these graphics cards in combination with graphics passthrough).<br />
<br />
The Nvidia Quadro 2000 graphics adapter has been reported to work with VGA passthrough, using a Xen 4.1.2 hypervisor. In addition, Nvidia lists the following graphics adapters as "Multi-OS" capable: Quadro 6000, 5000, 4000, 2000. "Multi-OS" allows VGA passthrough to fully virtualized guests.<br />
<br />
Nvidia Quadro 600 (and 450) are '''not''' supported.<br />
<br />
<br />
<pre><br />
Xen 4.0.0-rc8, pvops xen/stable-2.6.32.x dom0 kernel. Asus P7P55D MB, BIOS 1207, Core i7 860 CPU. <br />
Linux HVM guest with nv driver. Primary adapter. Passthru for single/only PCI ID. hidden from dom0: xen-pciback.hide=(01:00.0) <br />
<br />
01:00.0 VGA compatible controller: nVidia Corporation G92 [GeForce 8800 GTS 512] (rev a2)<br />
</pre><br />
<br />
== ATI/AMD display adapters ==<br />
AMD sent patch enabling VGA passthrough on AMD/ATI Radeon/FirePro graphics adapters on 2010-10-07: http://lists.xensource.com/archives/html/xen-devel/2010-10/msg00284.html .<br />
<br />
Tested adapters by AMD: <br />
<br />
* Radeon 4850<br />
* Radeon 4870<br />
* [[FirePro]] V5700<br />
* [[FirePro]] M5800<br />
<br />
User reported working adapters: <br />
<br />
* ATI HD4550<br />
* Radeon 4770 on Fedora 12, WinXP and Win7<br />
* ATI HD5750, using the Catalyst 10-9 driver on Win7 32bit<br />
* ATI HD5850<br />
* ATI HD6770 (tested with pciback under WinXP, xen 4.1.2 and 4.1.3)<br />
* ATI HD6850 under WinXP and Win7<br />
* ATI Radeon HD 6970 under Win7 64bit<br />
* AMD HD7970 under Win7<br />
* AMD HD7950 under Win7 64bit (Catalyst 12.10), Arch Linux 64bit (Catalyst 12.11) using XEN 4.2 (pciback)<br />
* AMD HD6870 under Win7 64bit using Xen 4.2(only with xl tools) <br />
<br />
There are certain user-reports that the Catalyst Control Center (ccc.exe) should be _not_ installed when installing the Catalyst Driver - for unknown reasons it may cause a BSOD. The GPU Driver itself works without problems when not installing CCC.<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel WSM-EP, Integrated graphics. <br />
RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0b:01.0). <br />
<br />
0b:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)<br />
</pre><br />
<br />
Xen 4.2 unstable changeset 25167 (Xen 4.1.2, 4.1.3, and many 4.2 changesets as well), Debian Wheezy, EFI grub, Kernel 3.3.2 (and 3.3.0), ATI Radeon HD 6870, with some difficulty installing the Catalyst drivers.<br />
<br />
== Matrox display adapters ==<br />
Tested adapters: MGA G200e<br />
<br />
<pre><br />
Xen 4.2-unstable changeset 25099, RedHat6.2 vmlinux-3.1.0-rc7 dom0 kernel. Intel Romley-EP, Integrated graphics. <br />
Windows8 Beta HVM guest, RedHat6.2 HVM guest Integrated graphics. Primary adapter. Passthru single PCI ID, hidden from dom0: xen-pciback.hide=(0a:00.0). <br />
<br />
0a:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G200e [Pilot] ServerEngines (SEP1) (rev 04)<br />
</pre><br />
<br />
<br />
== Links ==<br />
See [[XenVGAPassthrough]] wiki page for more information about Xen VGA graphics passthru in general.<br />
<br />
[[Category:Xen]]<br />
[[Category:Users]]<br />
[[Category:Developers]]<br />
[[Category:Compatibility]]<br />
[[Category:VGA]]</div>Powerhouse