Xen ARM with Virtualization Extensions/Allwinner: Difference between revisions

From Xen
Jump to navigationJump to search
m (add mkimage comand accordign to Roger Pau Monné<roger.pau@citrix.com>'s report)
(No difference)

Revision as of 19:05, 4 November 2013

this wiki talks about xen on A20(Cortex-A7 dual core, code name sun7i). A31(Cortex-A7 quad core, code name sun6i) should work as well.

Compile u-boot

as described in Firmware_Requirements, xen need boot from non-secure hyper mode.

get the code

a worked branch is here: 

   https://github.com/bjzhang/u-boot-sunxi/tree/sunxi_hyp

here is the details:

merge the hypmode branch
   git://git.linaro.org/people/aprzywara/u-boot.git hypmode_v4

branch into 

   git://github.com/linux-sunxi/u-boot-sunxi.git u-boot-sunxi

i tested hypmode_v4 with commit 3d2fc4e8 in u-boot-sunxi. hypmode_v5 should work as well.

add Ian Campbell patches on it.
enable arm generic timer for sunxi, xen dom0 kernel need this
   diff --git a/include/configs/sun7i.h b/include/configs/sun7i.h
   index f0d9222..473d1eb 100644
   --- a/include/configs/sun7i.h
   +++ b/include/configs/sun7i.h
   @@ -36,6 +36,8 @@
    
    #define CONFIG_ARMV7_VIRT
    
   +#define CONFIG_SYS_CLK_FREQ     24000000
   +
    #if defined(CONFIG_SYS_SECONDARY_ON)
    #define CONFIG_BOARD_POSTCLK_INIT 1
    #endif

refernce: bootwrapper can't be compiled for cubieboard2

Compile

for sd card 

   make Cubieboard2 CROSS_COMPILE=arm-linux-gnueabihf- -j8

for usb boot 

   make Cubieboard2_FEL CROSS_COMPILE=arm-linux-gnueabihf- -j8

Building Linux Dom0

First, we will build Linux for dom0 to have a device tree to provide to Xen.

You can get a tree from sun7i-xen-dom0 which is based on arokux sunxi usb branch. the emac patch in this tree is needed for enable network in dom0. i am confuse about what does "CONFIG XEN" mean. it will check the kernel features for dom0 and domU kernel. if you want to a dom0 kernel, you need to uncheck all the FRONTEND config, check CONFIG_XEN_BACKEND, and check the BACKEND config as you need: 

 CONFIG_XEN_BLKDEV_BACKEND=y
 CONFIG_XEN_NETDEV_BACKEND=y

or use sunxi dom0 kernel directly 

 git clone -b sun7i-xen-dom0 https://github.com/bjzhang/linux-allwinner.git sun7i-xen-dom0
 cd sun7i-xen-dom0
 make sun7i_dom0_defconfig
 make zImage
 make dtbs

The device tree is located in arch/arm/boot/dts/sun7i-a20-cubieboard2-xen.dtb

Building Xen

the lastest xen should work. reference Build Xen on ARM and Building Xen and Linux Dom0 for building xen and generating the xen-uImage. i test pass on xen commit 720f45ad: Julien Grall, xen/evtchn: Fix build on ARM

use the following build command for enable early printk for a20: 

   make dist-xen XEN_TARGET_ARCH=arm32 CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_EARLY_PRINTK=sun7i CONFIG_DTB_FILE=/path/to/sun7i-a20-cubieboard2-xen.dtb

and generate the xen image, note that sunxi ddr base physical base address is 0x40000000, do not use 0x80000000 for base address as other platform. 

   mkimage -A arm -T kernel -a 0x40200000 -e 0x40200000 -C none -d "xen/xen" xen-uImage

Building linux DomU

xen domU kernel is supported by upstream. and remember that domU guest do not aware of which hardware it running on, it only know about the mach-virt. if you want to a domU kernel, you need to ucheck all the BACKEND config and check the FRONTEND config as you need: 

 CONFIG_XEN_BLKDEV_FRONTEND=y
 CONFIG_XEN_NETDEV_FRONTEND=y
 CONFIG_HVC_XEN_FRONTEND=y
 CONFIG_XEN_FBDEV_FRONTEND=y

note that CONFIG_XEN_XENBUS_FRONTEND will be automatically checked by any FRONTEND driver.

currenly, xen directly use the dom0 gic dist base and cpu base as domU gic dist and cpu base. reference gicv_setup() in xen/arch/arm/gic.c for details. for sunxi, need chances the followings: (this patch is included in my branch) 

   diff --git a/arch/arm/boot/dts/xenvm-4.2.dts b/arch/arm/boot/dts/xenvm-4.2.dts
   index 3369151..5d5689e 100644
   --- a/arch/arm/boot/dts/xenvm-4.2.dts
   +++ b/arch/arm/boot/dts/xenvm-4.2.dts
   @@ -50,13 +50,13 @@
           reg = <0 0x80000000 0 0x08000000>;
       };
    
   -	gic: interrupt-controller@2c001000 {
   +	gic: interrupt-controller@1c81000{
           compatible = "arm,cortex-a15-gic", "arm,cortex-a9-gic";
           #interrupt-cells = <3>;
           #address-cells = <0>;
           interrupt-controller;
   -		reg = <0 0x2c001000 0 0x1000>,
   -		      <0 0x2c002000 0 0x100>;
   +		reg = <0 0x01c81000 0 0x1000>,
   +		      <0 0x01c82000 0 0x1000>;
       };
    
       timer {

it should not need to change after ian auto-generate dts patch upstreamed. 

 git clone -b sun7i_xen_domU https://github.com/bjzhang/linux-allwinner.git sun7i_xen_domU
 cd sun7i_xen_domU
 make sun7i_domU_defconfig
 make zImage
 make dtbs

Booting Xen and Dom0

To boot Xen and Dom0, you can use USB boot or sd boot.

Booting via PXE

TODO

Booting via USBBoot(FEL)

reference: http://linux-sunxi.org/FEL/USBBoot

Booting directly on the SD card

reference [1] for how to make a bootable sdcard.

  • Copy xen-uImage, zImage and the initrd the root directory of you SD card.
  • save the bootcmd
 setenv bootcmd "fatload mmc 0 45000000 xen-uImage; fatload mmc 0 50000000 zImage; fatload mmc 0 60000000 initrd; bootm 45000000"
 savenv
  • boot
 boot

Assume the SD card has only 1 partition with fat filesystem.

Mount NFS after initrd boot

Using a distrbution is easily to compile xen tools. i use opensuse arm 12.3 rootfs for it. other distribution which support armhf is ok too.

Create domU

setup the harddisk
   losetup /dev/loop0 /root/domU/rootfs.ext3
domU config example
   > cat domU_test
   kernel = "/root/domU/zImage_dtb" 
   memory = "128" 
   name = "domU" 
   vcpus = 1 
   serial="pty"
   disk = [ 'phy:/dev/loop0,xvda,w' ]
   vif=[ 'mac=00:16:3e:56:af:69,bridge=virbr0,type=netfront', ]
create
   xl create domU_test

or enable all the debug message 

   xl -vvv create -d domU_testa
connect to console

xl console domU_test

DomU kernel debug

xen support varity debug method. you could use xl debug-keys for dumping information. use xen_raw_console_write for output DomU kernel output. but gdbsx is not work for arm right now. 

   diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c
   index b4e8500..060c163 100644
   --- a/kernel/printk/printk.c
   +++ b/kernel/printk/printk.c
   @@ -1488,6 +1488,7 @@ static size_t cont_print_text(char *text, size_t size)
       return textlen;
    }
    
   +#include <xen/hvc-console.h>
    asmlinkage int vprintk_emit(int facility, int level,
                   const char *dict, size_t dictlen,
                   const char *fmt, va_list args)
   @@ -1546,6 +1547,7 @@ asmlinkage int vprintk_emit(int facility, int level,
        * prefix which might be passed-in as a parameter.
        */
       text_len = vscnprintf(text, sizeof(textbuf), fmt, args);
   +    xen_raw_console_write(text);
    
       /* mark and strip a trailing newline */
       if (text_len && text[text_len-1] == '\n') {

Resources

allwinner sunxi resources [2]

Known Issue

without sunxi platform patch, there is a warning during xen boot: 

 (XEN) WARNING: Unrecognized/unsupported device tree compatible list
 (XEN) Available platform support:
 (XEN)     - VERSATILE EXPRESS
 (XEN)     - SAMSUNG EXYNOS5
 (XEN)     - CALXEDA MIDWAY
 (XEN)     - TI OMAP5
Retrieved from "https://wiki.xenproject.org/index.php?title=Xen_ARM_with_Virtualization_Extensions/Allwinner&oldid=9978"