Xen ARM with Virtualization Extensions/Allwinner
this wiki talks about xen on A20(Cortex-A7 dual core, code name sun7i). A31(Cortex-A7 quad core, code name sun6i) should work as well.
Compile u-boot
as described in Firmware_Requirements, xen need boot from non-secure hyper mode.
get the code
a worked branch is here:
https://github.com/bjzhang/u-boot-sunxi/tree/sunxi_hyp
here is the details:
- merge the hypmode branch
git://git.linaro.org/people/aprzywara/u-boot.git hypmode_v4
branch into
git://github.com/linux-sunxi/u-boot-sunxi.git u-boot-sunxi
i tested hypmode_v4 with commit 3d2fc4e8 in u-boot-sunxi. hypmode_v5 should work as well.
- add Ian Campbell patches on it.
- enable arm generic timer for sunxi, xen dom0 kernel need this
diff --git a/include/configs/sun7i.h b/include/configs/sun7i.h
index f0d9222..473d1eb 100644
--- a/include/configs/sun7i.h
+++ b/include/configs/sun7i.h
@@ -36,6 +36,8 @@
#define CONFIG_ARMV7_VIRT
+#define CONFIG_SYS_CLK_FREQ 24000000
+
#if defined(CONFIG_SYS_SECONDARY_ON)
#define CONFIG_BOARD_POSTCLK_INIT 1
#endif
refernce: bootwrapper can't be compiled for cubieboard2
Compile
for sd card
make Cubieboard2 CROSS_COMPILE=arm-linux-gnueabihf- -j8
for usb boot
make Cubieboard2_FEL CROSS_COMPILE=arm-linux-gnueabihf- -j8
Building Linux Dom0
First, we will build Linux for dom0 to have a device tree to provide to Xen.
You can get a tree from sun7i-xen-dom0 which is based on arokux sunxi usb branch. the emac patch in this tree is needed for enable network in dom0. i am confuse about what does "CONFIG XEN" mean. it will check the kernel features for dom0 and domU kernel. if you want to a dom0 kernel, you need to uncheck all the FRONTEND config, check CONFIG_XEN_BACKEND, and check the BACKEND config as you need:
CONFIG_XEN_BLKDEV_BACKEND=y CONFIG_XEN_NETDEV_BACKEND=y
or use sunxi dom0 kernel directly
git clone -b sun7i-xen-dom0 https://github.com/bjzhang/linux-allwinner.git sun7i-xen-dom0 cd sun7i-xen-dom0 make sun7i_dom0_defconfig make zImage make dtbs
The device tree is located in arch/arm/boot/dts/sun7i-a20-cubieboard2-xen.dtb
Building Xen
the lastest xen should work. reference Build Xen on ARM and Building Xen and Linux Dom0 for building xen and generating the xen-uImage. i test pass on xen commit 720f45ad: Julien Grall, xen/evtchn: Fix build on ARM
use the following build command for enable early printk for a20:
make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=sun7i CONFIG_DTB_FILE=/path/to/sun7i-a20-cubieboard2-xen.dtb
and generate the xen image, note that sunxi ddr base physical base address is 0x40000000, do not use 0x80000000 for base address as other platform.
mkimage -A arm -T kernel -a 0x40200000 -e 0x40200000 -C none -d "xen/xen" xen-uImage
Building linux DomU
xen domU kernel is supported by upstream. and remember that domU guest do not aware of which hardware it running on, it only know about the mach-virt. if you want to a domU kernel, you need to ucheck all the BACKEND config and check the FRONTEND config as you need:
CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_HVC_XEN_FRONTEND=y CONFIG_XEN_FBDEV_FRONTEND=y
note that CONFIG_XEN_XENBUS_FRONTEND will be automatically checked by any FRONTEND driver.
currenly, xen directly use the dom0 gic dist base and cpu base as domU gic dist and cpu base. reference gicv_setup() in xen/arch/arm/gic.c for details. for sunxi, need chances the followings: (this patch is included in my branch)
diff --git a/arch/arm/boot/dts/xenvm-4.2.dts b/arch/arm/boot/dts/xenvm-4.2.dts
index 3369151..5d5689e 100644
--- a/arch/arm/boot/dts/xenvm-4.2.dts
+++ b/arch/arm/boot/dts/xenvm-4.2.dts
@@ -50,13 +50,13 @@
reg = <0 0x80000000 0 0x08000000>;
};
- gic: interrupt-controller@2c001000 {
+ gic: interrupt-controller@1c81000{
compatible = "arm,cortex-a15-gic", "arm,cortex-a9-gic";
#interrupt-cells = <3>;
#address-cells = <0>;
interrupt-controller;
- reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x100>;
+ reg = <0 0x01c81000 0 0x1000>,
+ <0 0x01c82000 0 0x1000>;
};
timer {
it should not need to change after ian auto-generate dts patch upstreamed.
git clone -b sun7i_xen_domU https://github.com/bjzhang/linux-allwinner.git sun7i_xen_domU cd sun7i_xen_domU make sun7i_domU_defconfig make zImage make dtbs
Booting Xen and Dom0
To boot Xen and Dom0, you can use USB boot or sd boot.
Booting via PXE
TODO
Booting via USBBoot(FEL)
reference: http://linux-sunxi.org/FEL/USBBoot
Booting directly on the SD card
reference [1] for how to make a bootable sdcard.
- Copy xen-uImage, zImage and the initrd the root directory of you SD card.
- save the bootcmd
setenv bootcmd "fatload mmc 0 45000000 xen-uImage; fatload mmc 0 50000000 zImage; fatload mmc 0 60000000 initrd; bootm 45000000" savenv
- boot
boot
Assume the SD card has only 1 partition with fat filesystem.
Mount NFS after initrd boot
Using a distrbution is easily to compile xen tools. i use opensuse arm 12.3 rootfs for it. other distribution which support armhf is ok too.
Create domU
- setup the harddisk
losetup /dev/loop0 /root/domU/rootfs.ext3
- domU config example
> cat domU_test kernel = "/root/domU/zImage_dtb" memory = "128" name = "domU" vcpus = 1 serial="pty" disk = [ 'phy:/dev/loop0,xvda,w' ] vif=[ 'mac=00:16:3e:56:af:69,bridge=virbr0,type=netfront', ]
- create
xl create domU_test
or enable all the debug message
xl -vvv create -d domU_testa
- connect to console
xl console domU_test
DomU kernel debug
xen support varity debug method. you could use xl debug-keys for dumping information. use xen_raw_console_write for output DomU kernel output. but gdbsx is not work for arm right now.
diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
index b4e8500..060c163 100644
--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -1488,6 +1488,7 @@ static size_t cont_print_text(char *text, size_t size)
return textlen;
}
+#include <xen/hvc-console.h>
asmlinkage int vprintk_emit(int facility, int level,
const char *dict, size_t dictlen,
const char *fmt, va_list args)
@@ -1546,6 +1547,7 @@ asmlinkage int vprintk_emit(int facility, int level,
* prefix which might be passed-in as a parameter.
*/
text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
+ xen_raw_console_write(text);
/* mark and strip a trailing newline */
if (text_len && text[text_len-1] == '\n') {
Resources
Known Issue
without sunxi platform patch, there is a warning during xen boot:
(XEN) WARNING: Unrecognized/unsupported device tree compatible list (XEN) Available platform support: (XEN) - VERSATILE EXPRESS (XEN) - SAMSUNG EXYNOS5 (XEN) - CALXEDA MIDWAY (XEN) - TI OMAP5
