HiKey960

Xen ARM64 on 96boards HiKey960

NOTE that Hikey 960 is very different than Hikey! DO NOT TRY TO USE INSTRUCTIONS OR BINARIES FOR HIKEY

Xen is now supported upstream on the 96boards HiKey, however beware that to run Xen on HiKey you need the 96Boards UART Serial Adapter, available here, or here.

Requirements

To use Xen under this board you need to flash it with EDK2 (TianoCore) so it boots in UEFI mode. You can then boot Xen using either method:

• GRUB
• UEFI Shell (bundled in UEFI)

The following git trees and branches are known to work:

• https://github.com/96boards-hikey/edk2.git #testing/hikey960_v2.5
• https://github.com/96boards-hikey/OpenPlatformPkg.git #testing/hikey960_v1.3.4
• https://git.savannah.gnu.org/git/grub.git #master
• https://github.com/96boards-hikey/linux.git #hikey960-upstream-rebase
• https://github.com/96boards-hikey/tools-images-hikey960.git #master
• git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git #master

Flashing the Hikey 960 is using the fastboot mechanism. This guide explains how to build (or get pre-built) the:

• ptable - aka Partition Table layout for the 32GB NAND
• xloader - the initial secure mode loader
• fastboot - the image with fastboot and UEFI firmware
• fip - No idea
• boot - The FAT32 64MB partition containing GRUB (or xen.efi if desired)
• system - The rootfs (Ext4) containing the OS and /boot directory for GRUB to search.

You can retrieve rootfs from Latest RPB , look for rpb-console-image-hikey960-<some number>.rootfs.img.gz Linaro latest ARM64 developer images can be used as well.

And also you need the boot-0.0+AUTOINC+<some number>hikey960-<some number>.uefi.img from Latest RPB

Keep in mind that the version of Linux that is on this web-site is not built with Xen support so you will need to build Linux from scratch (see below).

Building UEFI

All the files needed are pre-built and available at Debug. The instructions are (if you prefer to build it from source) are available at README - ATF UEFI build from source.

If you are downloading the binaries already pre-built - then start in instructions at Boot UEFI in recovery mode (and yes it makes no sense as you do not have UEFI installed). The recovery mode is described in Step 3 Enter recovery forced download mode.

Follow exactly the steps - especially the Prepare the config file.

Before you flash the boot.img (boot-0.0.. so on) and system.img (rpb-console-<something>rootfs.img.gz it is recommended you modify these two images. See below.

Building Linux

You need to build Linux before Xen or GRUB as you will need the DTB file to seed GRUB2. The prebuilt version of Linux that is available at Latest RPB is not built with CONFIG_XEN so you need to compile you own.

Fortunately the instructions are simple

git clone https://github.com/96boards-hikey/linux.git
cd linux
git checkout origin/hikey960-upstream-rebase
export CROSS_COMPILE=aarch64-linux-gnu-
export ARCH=arm64
make defconfig
make -j8
cp -f arch/$ARCH/boot/Image ../Image
cp -f arch/$ARCH/boot/dts/hisilicon/hi3660-hikey960.dtb ../Image.dtb

The Image and Image.dtb will be put in the rootfs you have downloaded.

Building Xen

Upstream Xen is to be used:

git clone git://xenbits.xen.org/xen.git
cd xen/xen
export CROSS_COMPILE=aarch64-linux-gnu-
export ARCH=arm64
export XEN_TARGET_ARCH=arm64
make defconfig
make -j8
cp xen.efi ../../

The xen.efi will be put in the rootfs you have downloaded.

Extract the wl18xx-fw-4.bin so your WiFi will work

git clone git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
cd linux-firmware
cp ti-connectivity/wl18xx-fw-4.bin ../

The wl18xx-fw-4.bin will be used in your rootfs you have downloaded.

Modify rootfs

Use any rootfs you wish. For example Latest Linaro ARM64 developer can be used.

https://releases.linaro.org/debian/images/developer-arm64/latest/linaro-stretch-developer-20170720-71.tar.gz
sudo tar -xzf linaro-stretch-developer-20170720-71.tar.gz
sudo mv binary rootfs
sudo cp Image rootfs/boot
sudo cp Image.dtb rootfs/boot
sudo cp xen.efi rootfs/boot
sudo mkdir -p rootfs/lib/firmware/ti-connectivity
sudo cp wl18xx-fw-4.bin rootfs/lib/firmware/ti-connectivity

Modify the rootfs/boot/grub.cfg to your satisfaction (see below in Building GRUB section).

Create new rootfs

mkdir loop
dd if=/dev/zero of=rootfs.img bs=1M count=2096
mkfs.ext4 -F -L rootfs rootfs.img #kernel use the label rootfs to mount the rootfs as /
sudo mount -o loop rootfs.img loop
cd rootfs
sudo tar -cf - * | (cd ../loop;sudo tar -xf - )
sudo umount loop
img2simg rootfs.img rootfs.sparse.img 4096

Whenever the instructions asks you to fastboot flash system system.img instead of system.img use rootfs.sparse.img

Building GRUB

The upstream version of GRUB #master has the patches to boot Xen hypervisor using the xen_boot module.

The stanza you will need to boot Xen is very simple:

 menuentry 'Xen' {
     xen_hypervisor /boot/xen.efi loglvl=all console=dtuart dtuart=/soc/serial@fff32000 efi=no-rs
     xen_module /boot/Image console=tty0 console=hvc0 root=/dev/sdd10 rw efi=noruntime
     devicetree /boot/Image.dtb
 }

But the version of GRUB which is in the boot.img is an older version. Hence you need to build it from scratch.

git clone  https://git.savannah.gnu.org/git/grub.git
cd grub
export CROSS_COMPILE=aarch64-linux-gnu-
export ARCH=arm64
./configure --prefix=/usr --target=aarch64-linux-gnu --with-platform=efi
make
mkdir ../grub-install
make DESTDIR=../grub-install install
cd ../

The next step is to create grubaa64.efi. You create it by providing to grub-mkimage a list of modules. You need xen_boot but also other ones.

Also you need to provide an embedded configuration file grub.config

$cat grub.cfg
search.fs_label rootfs root

set prefix=($root)/boot/grub
configfile $prefix/grub.cfg

NOTE. The fs.label will search for a partition called rootfs. It should match exactly with what you had called it when you invoked mkfs.ext4 .. -L rootfs.

To create grubaa64.efi:

GRUB_MODULES="boot chain configfile echo efinet eval ext2 fat font gettext gfxterm gzio help linux loadenv lsefi normal part_gpt part_msdos read regexp search search_fs_file search_fs_uuid search_label terminal terminfo test tftp time xen_boot"
grub-install/usr/bin/grub-mkimage 
               --config grub.config \
               --dtb Image.dtb \
               --directory=`pwd`/grub-install/usr/lib64/grub/arm64-efi \
               --output=grubaa64.efi \
               --format=arm64-efi \
               --prefix="/boot/grub" \
               $GRUB_MODULES

Update boot.img

sudo mount boot-0.0+AUTOINC+<some number>hikey960-<some number>.uefi.img loop -o loop
cp grubaa64.efi loop/EFI/BOOT
sudo umount loop

Flash boot.img and system.img

If you were following README ATF UEFI Build from source you should replace the calls of:

$sudo fastboot flash boot boot.img

$sudo fastboot flash system system.img

with:

$sudo fastboot flash boot boot-0.0+AUTOINC+<some number>hikey960-<some number>.uefi.img 
$sudo fastboot flash system rootfs.sparse.img

Once the flash is complete, and you have moved the device to Normal boot and rebooted the device, you will be greeted with:

                          GNU GRUB  version 2.03
/----------------------------------------------------------------------------\
|*Xen                                                                        | 
| CE Reference Platform (HiKey960 rpb)                                       |
| Fastboot                                                                   |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            |
|                                                                            | 
\----------------------------------------------------------------------------/
     Use the ^ and v keys to select which entry is highlighted.          
     Press enter to boot the selected OS, `e' to edit the commands       
     before booting or `c' for a command-line. ESC to return previous    
     menu.                                                               
                                                                              

Install the rest of Xen (tools, etc)

At this point you are booted in and should be able to use the WiFi. The instructions at Networking CLI should provide the guidance on how to setup WiFi using the command line.

Once you have the network you can download Xen and built the toolstack natively on your Hikey 960!