Difference between revisions of "Xen ARM with Virtualization Extensions/qemu-system-aarch64"

From Xen
 
(5 intermediate revisions by 2 users not shown)
Line 7: Line 7:
 
== Build QEMU ==
 
== Build QEMU ==
   
QEMU v2.11 can run Xen out of the box. Build QEMU like this:
+
QEMU v4.0.0 can run Xen out of the box. Build QEMU like this:
   
$ ./configure --target-list=aarch64-softmmu
+
$ ./configure --target-list=aarch64-softmmu --prefix=/usr/local
 
$ make
 
$ make
  +
$ sudo make install
   
 
== Install Xen ==
 
== Install Xen ==
   
 
QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like [https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-uefi1.img xenial-server-cloudimg-arm64-uefi1.img].
 
QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like [https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-uefi1.img xenial-server-cloudimg-arm64-uefi1.img].
  +
 
Then, clone Linux and build the kernel using the defconfig for arm64 (make defconfig && make Image.gz).
 
Then, clone Linux and build the kernel using the defconfig for arm64 (make defconfig && make Image.gz).
  +
For building Linux kernel for ARM64 / ARM user might need cross compilers. Below are details with respect to different distros :
  +
  +
opensuse : https://opensuse.pkgs.org/tumbleweed/opensuse-oss/cross-aarch64-gcc7-7.3.1+r258313-1.2.x86_64.rpm.html
  +
  +
ubuntu : https://packages.ubuntu.com/en/trusty/gcc-aarch64-linux-gnu
  +
  +
Other distros : https://pkgs.org/
  +
  +
Steps can be found at http://events17.linuxfoundation.org/sites/events/files/slides/Shuah_Khan_cross_compile_linux.pdf
   
 
Run QEMU the first time, booting Linux directly (replace the MAC_ADDRESS and the paths). The following command uses user-networking and forwards port 2222 from the host to port 22 inside the virtual machine:
 
Run QEMU the first time, booting Linux directly (replace the MAC_ADDRESS and the paths). The following command uses user-networking and forwards port 2222 from the host to port 22 inside the virtual machine:
   
$ /path/to/qemu.git/aarch64-softmmu/qemu-system-aarch64 \
+
$ qemu-system-aarch64 \
 
-machine virt,gic_version=3 -machine virtualization=true \
 
-machine virt,gic_version=3 -machine virtualization=true \
 
-cpu cortex-a57 -machine type=virt -nographic \
 
-cpu cortex-a57 -machine type=virt -nographic \
 
-smp 4 -m 4000 \
 
-smp 4 -m 4000 \
-kernel /path/to/linux.git/arch/arm64/boot/Image.gz --append "console=ttyAMA0 root=/dev/vda1" \
+
-kernel /path/to/linux.git/arch/arm64/boot/Image.gz --append "console=ttyAMA0 root=/dev/vda1 init=/bin/sh" \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
 
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0
 
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0
  +
  +
This command will give a shell prompt, where user might have to mount the root file system using below command :
  +
  +
$ mount -o remount,rw /dev/vda1 /
  +
  +
Once mounted, user would like to set the 'root' password using 'passwd' command and then relaunch the virtual machine using above qemu command, but now remove 'init=/bin/sh'.
  +
  +
Once login with 'root' user, you may need to set the ssh authorized_keys of host machine, to access the virtual machine via remote.
   
 
Follow [[Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM]] to build Xen on ARM. Then, copy the Xen on ARM efi binary and the Linux kernel to the image:
 
Follow [[Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM]] to build Xen on ARM. Then, copy the Xen on ARM efi binary and the Linux kernel to the image:
Line 44: Line 63:
 
Now we need the device tree binary, "virt-gicv3.dtb". QEMU can generate one for you with the following command:
 
Now we need the device tree binary, "virt-gicv3.dtb". QEMU can generate one for you with the following command:
   
$ /path/to/qemu.git/aarch64-softmmu/qemu-system-aarch64 \
+
$ qemu-system-aarch64 \
 
-machine virt,gic_version=3 \
 
-machine virt,gic_version=3 \
 
-machine virtualization=true \
 
-machine virtualization=true \
 
-cpu cortex-a57 -machine type=virt \
 
-cpu cortex-a57 -machine type=virt \
 
-smp 4 -m 4096 -display none \
 
-smp 4 -m 4096 -display none \
-machine dumpdtb=virt-gicv3.dtb.dtb
+
-machine dumpdtb=virt-gicv3.dtb
$ scp -P2222 virt-gicv3.dtb.dtb root@127.0.0.1:/boot/efi
+
$ scp -P2222 virt-gicv3.dtb root@127.0.0.1:/boot/efi
   
 
We have written everything we need to the disk image. Let's poweroff the virtual machine and proceed to download the UEFI firmware binary from Linaro: [http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_CLANG35/QEMU_EFI.fd].
 
We have written everything we need to the disk image. Let's poweroff the virtual machine and proceed to download the UEFI firmware binary from Linaro: [http://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/latest/QEMU-AARCH64/RELEASE_CLANG35/QEMU_EFI.fd].
 
   
 
== Run QEMU ==
 
== Run QEMU ==
Line 59: Line 77:
 
The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM, booting from EFI. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.
 
The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM, booting from EFI. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.
   
$ /path/to/qemu.git/aarch64-softmmu/qemu-system-aarch64 \
+
$ qemu-system-aarch64 \
 
-machine virt,gic_version=3 \
 
-machine virt,gic_version=3 \
 
-machine virtualization=true \
 
-machine virtualization=true \
Line 67: Line 85:
 
-bios /path/to/QEMU_EFI.bin \
 
-bios /path/to/QEMU_EFI.bin \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
 
-netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0
+
-drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0 -boot order=d
  +
  +
The boot order option will enable to get to UEFI prompt. Typing "FS0:" and then "xen" will boot hypervisor.
  +
  +
The Xen tools can be built further using below links :
  +
  +
https://wiki.debian.org/Arm64Qemu
  +
https://wiki.debian.org/QemuUserEmulation "Appendix: chrooting into target file systems"
  +
  +
= External References =
  +
  +
There is a nice [https://web.eecs.umich.edu/~jcma/blog/cjrl7dk7d000brlqjrv1rt7ru/ blog] entry covering the same topic.

Latest revision as of 12:46, 24 April 2019

QEMU AArch64 Emulator

QEMU is an Open Source GPLv2 software emulator. It can emulate a large range of machines of different architectures, including Cortex A57s based platforms.

The following steps help you setup QEMU to emulate an ARM64 machine and run Xen inside it.

Build QEMU

QEMU v4.0.0 can run Xen out of the box. Build QEMU like this:

$ ./configure --target-list=aarch64-softmmu --prefix=/usr/local
$ make
$ sudo make install

Install Xen

QEMU works well with UEFI firmware for the emulated machine. To get the system working, download an Aarch64 UEFI ready distro image, like xenial-server-cloudimg-arm64-uefi1.img.

Then, clone Linux and build the kernel using the defconfig for arm64 (make defconfig && make Image.gz). For building Linux kernel for ARM64 / ARM user might need cross compilers. Below are details with respect to different distros :

opensuse : https://opensuse.pkgs.org/tumbleweed/opensuse-oss/cross-aarch64-gcc7-7.3.1+r258313-1.2.x86_64.rpm.html

ubuntu : https://packages.ubuntu.com/en/trusty/gcc-aarch64-linux-gnu

Other distros : https://pkgs.org/

Steps can be found at http://events17.linuxfoundation.org/sites/events/files/slides/Shuah_Khan_cross_compile_linux.pdf

Run QEMU the first time, booting Linux directly (replace the MAC_ADDRESS and the paths). The following command uses user-networking and forwards port 2222 from the host to port 22 inside the virtual machine:

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt -nographic \
   -smp 4 -m 4000 \
   -kernel /path/to/linux.git/arch/arm64/boot/Image.gz --append "console=ttyAMA0 root=/dev/vda1 init=/bin/sh" \
   -netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
   -drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0

This command will give a shell prompt, where user might have to mount the root file system using below command :

 $ mount -o remount,rw /dev/vda1 /

Once mounted, user would like to set the 'root' password using 'passwd' command and then relaunch the virtual machine using above qemu command, but now remove 'init=/bin/sh'.

Once login with 'root' user, you may need to set the ssh authorized_keys of host machine, to access the virtual machine via remote.

Follow Xen_ARM_with_Virtualization_Extensions#Building_Xen_on_ARM to build Xen on ARM. Then, copy the Xen on ARM efi binary and the Linux kernel to the image:

 $ scp -P2222 /path/to/linux.git/arch/arm64/boot/Image.gz root@127.0.0.1:/boot/efi/kernel
 $ scp -P2222 /path/to/xen.git/xen/xen.efi root@127.0.0.1:/boot/efi

Let's write a config file for it:

 $ vi /boot/efi/xen.cfg

Let's use the following config:

 options=console=dtuart noreboot dom0_mem=512M
 kernel=kernel root=/dev/vda1 init=/bin/sh rw console=hvc0
 dtb=virt-gicv3.dtb

Now we need the device tree binary, "virt-gicv3.dtb". QEMU can generate one for you with the following command:

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 \
   -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt \
   -smp 4 -m 4096 -display none \
   -machine dumpdtb=virt-gicv3.dtb
 $ scp -P2222 virt-gicv3.dtb root@127.0.0.1:/boot/efi

We have written everything we need to the disk image. Let's poweroff the virtual machine and proceed to download the UEFI firmware binary from Linaro: [1].

Run QEMU

The following command create a new emulated AArch64 machine with 4 Cortex A57s, 4G of RAM, booting from EFI. The command below uses user networking in QEMU, other network configuration (bridging) are possible. Please change MAC_ADDRESS for the emulated machine and path to the guest image.

 $ qemu-system-aarch64 \
   -machine virt,gic_version=3 \
   -machine virtualization=true \
   -cpu cortex-a57 -machine type=virt \
   -smp 4 -m 4096 -display none \
   -serial mon:stdio \
   -bios /path/to/QEMU_EFI.bin \
   -netdev user,id=hostnet0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=hostnet0,mac=MAC_ADDRESS \
   -drive if=none,file=/path/to/xenial-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0 -boot order=d

The boot order option will enable to get to UEFI prompt. Typing "FS0:" and then "xen" will boot hypervisor.

The Xen tools can be built further using below links :

https://wiki.debian.org/Arm64Qemu https://wiki.debian.org/QemuUserEmulation "Appendix: chrooting into target file systems"

External References

There is a nice blog entry covering the same topic.