Xen ARM with Virtualization Extensions/APMXGeneMustang
Xen ARM64 on APM XGENE Mustang
Xen is now supported upstream on the APM X-Gene Mustang platform.
Booting with U-Boot
The current approach requires tftp to download the software images to the board.
Basic IP address settings
setenv ipaddr 10.y.y.y #This is an IP address of your board. setenv serverip 10.x.x.x #This is an IP address of your tftp server. setenv ethaddr 00:zz:zz:zz:zz:zz #This is the MAC of your Ethernet. setenv user_dir <SERVER TFTP DIRECTORY FOR GETTING XEN IMAGES> #See below setenv xen_addr_r 0x4000080000 #This is the address where Xen kernel will be copied using tftp. setenv gatewayip 10.x.x.x setenv netmask 255.x.x.x ping $serverip #This should work before proceeding further. setenv kern_addr_r 0x4002000000 #This is the address where DOM0 kernel will be copied using tftp. setenv fdt_addr_r 0x4003000000 #This is the address where DOM0 dtb will be copied using tftp. saveenv
Where:
- <SERVER TFTP DIRECTORY>: e.g. If your images are inside /tftpboot/xen and /tftpboot is the tftp server root then "setenv user_dir xen"
Setting up Xen booting environment
setenv xen_load 'tftp ${xen_addr_r} ${user_dir}/uXen'
setenv xen_dtb 'tftp ${fdt_addr_r} ${user_dir}/apm-mustang-xen.dtb;fdt addr ${fdt_addr_r};fdt resize;'
setenv xen_boot 'setenv bootargs ${bootargs_xen};bootm ${xen_addr_r} - ${fdt_addr_r}'
setenv dom0_node 'fdt set /chosen/module@0 compatible xen,linux-zimage xen,multiboot-module'
setenv xen_dom0 'tftp ${kern_addr_r} ${user_dir}/Image; run dom0_node;fdt set /chosen/module@0 reg < 0x40 ${kern_addr_r} 0x${filesize} >'
setenv xen_run 'run xen_dtb xen_dom0 xen_load;fdt print /chosen;run xen_boot'
setenv bootargs_xen 'conswitch=x sync_console dom0_mem=1G dom0_max_vcpus=1 console=dtuart dtuart=/soc/serial@1c020000'
saveenv
Preparing all the Images
Building Xen Kernel
First, pull the latest xen code from:
http://xenbits.xen.org/gitweb/?p=xen.git;a=summary
Then, build the Xen kernel:
# make dist-xen XEN_TARGET_ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu-
TIP: To get early prints:
# make dist-xen XEN_TARGET_ARCH=arm64 debug=y CONFIG_EARLY_PRINTK=xgene-storm CROSS_COMPILE=aarch64-linux-gnu-
To create the image, do:
# mkimage -A arm -C none -T kernel -a 0x0200000 -e 0x00200000 -n Xen -d xen/xen uXen
Finally copy "uXen" to your tftp directory.
Building DOM0
For Dom0 rootfs we need SATA support to be enabled in the Dom0 kernel. At the time of writing (17/02/2014), SATA, CLOCK and PCIe Linux driver patches for Mustang are not yet merged and are under review. Hence please use Ian Campbell's linux tree which has un-merged SATA CLOCK and PCIe drivers for mustang. Once above mentioned drivers are made it to mainline then Dom0 can be build directly from mainline Linux.
To download and build Dom0 and DomU, do:
# git clone git://xenbits.xen.org/people/ianc/linux.git linux-dom0 # cd linux-dom0 # git checkout -b for-apm/mustang-v3.12-v3 origin/for-apm/mustang-v3.12-v3 # make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig # make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- menuconfig # make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image
Once completed, copy arch/arm64/boot/Image to your tftp directory.
NOTE: Please make sure you have following CONFIG options selected to "y" while compiling DOM0 kernel:
CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_BLKDEV_BACKEND=y CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_XEN_NETDEV_BACKEND=y CONFIG_HVC_DRIVER=y CONFIG_HVC_IRQ=y CONFIG_HVC_XEN=y CONFIG_HVC_XEN_FRONTEND=y CONFIG_XEN_BALLOON=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XEN_BACKEND=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_GNTDEV=y CONFIG_SWIOTLB_XEN=y CONFIG_XEN_PRIVCMD=y CONFIG_CLKDEV_LOOKUP=y CONFIG_HAVE_CLK_PREPARE=y CONFIG_COMMON_CLK=y CONFIG_BLK_DEV_LOOP=y CONFIG_BLK_DEV_LOOP_MIN_COUNT=8 CONFIG_ARCH_XGENE=y CONFIG_GENERIC_PHY=y CONFIG_PCI_XGENE=y CONFIG_SATA_AHCI=y CONFIG_SATA_AHCI_PLATFORM=y CONFIG_SATA_XGENE_PHY=y CONFIG_SATA_XGENE=y CONFIG_POWER_RESET_XGENE=y
You also need to apply the following patch for the Dom0 DTB:
diff --git a/arch/arm64/boot/dts/apm-mustang-xen.dts b/arch/arm64/boot/dts /apm-mustang-xen.dts
index 848b34a..d6ce7ee 100644
--- a/arch/arm64/boot/dts/apm-mustang-xen.dts
+++ b/arch/arm64/boot/dts/apm-mustang-xen.dts
@@ -18,6 +18,8 @@
compatible = "apm,mustang", "apm,xgene-storm";
chosen {
+ #address-cells=<2>;
+ #size-cells=<1>;
module@0 {
bootargs = "console=hvc0 root=/dev/sda2 rw earlyprintk=uart8250-32bit,0x1c020000 debug maxcpus=8"; /* initcall_debug*/
};
NOTE: Above bootargs expects that rootfs will reside on /dev/sda2 of your SATA drive.
To build the device tree blob (DTB), do:
# make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- dtbs # Copy arch/arm64/boot/dts/apm-mustang-xen.dtb to your tftpboot directory.
Preparing SATA Drive
Assuming we have a SATA disk with 2nd partition empty and formatted with ext3 or ext4, do:
# wget http://download.opensuse.org/repositories/devel:/ARM:/AArch64:/12.3/images/openSUSE-12.3-AArch64-JeOS-rootfs.aarch64-1.12.1-Build5.13.tbz
Then mount the 2nd SATA partition and untar the above rootfs on the 2nd SATA partition
Booting DOM0 Kernel
- Check your tftp directory has following files:
- uXen
- Image
- apm-mustang-xen.dts
- Make sure SATA disk (with above mantioned rootfs) is attached to mustang board.
- Make sure PCIe NiC (like Intel's e1000) is attached to your board and it's driver is enabled in DOM0 config (for e1000 CONFIG_E1000=y CONFIG_E1000E=y should be enabled).
On u-boot prompt execute:
# run xen_run
This should reach to DOM0 login prompt!!
DomU
Make sure your e1000 network card is up in DOM0 and has internet connectivity.
Instal necessary build packages like gcc, libfdt etc... (This is one time configuration after first boot):
# zypper update # zypper addrepo http://download.opensuse.org/repositories/devel:/ARM:/AArch64:/12.3/standard/devel:ARM:AArch64:12.3.repo # zypper install gcc make patterns-openSUSE-devel_basis extra packages: git vim libyajl-devel python-devel wget libfdt1-devel # cp libfdt_env.h /usr/include/
libfdt_env.h -> Can be found in DOM0 linux source at "scripts/dtc/libfdt/" folder.
Build xen tools
# git clone git://xenbits.xen.org/xen.git # cd xen # ./configure # make tools # make install
DomU Images
- Image : Kernel Image (same DOM0 kenel Image can be used here)
- rootfs.ext3 : Create ext3 rootdisk using an opensuse or linaro arm64 rootfs.
Booting DomU
First, setup the disk:
# losetup /dev/loop0 /root/domU/rootfs.ext3 # cat d name = "d" uuid = "3fb78ba6-8182-484c-acf7-8faba9773f68" disk = ["phy:/dev/loop0,xvda,rw"] memory = 512 kernel = "./Image" extra = "root=/dev/xvda rw console=hvc0 earlyprintk=xen" # /etc/init.d/xencommons start || true # echo 9 > /proc/sysrq-trigger # export LIBXL_DEBUG_DUMP_DTB=guest.dtb # xl -vvv create d # xl console d
Author
- This guide was written by Pranavkumar Sawargaonkar (pranavkumar _at_ linaro _dot_ org).
- Modified to the Xen wiki by: Christoffer Dall (christoffer _dot_ dall _at_ linaro _dot_ org)
