Tuning Xen for Performance

Tuning your Xen installation: recommended settings

Storage options

See Storage_options.

Memory

If the dom0 Host-to-be has more memory than a typical laptop/desktop system has, then do not rely on dom0 ballooning. Instead set the dom0 ram to something between 1 and 4GB adding dom0_mem=2048M to the Xen command line.

1GB is enough for a pretty large host, more will be needed if you expect your users to use advanced storage types as ZFS or distributed filesystems.

The reason to not give all ram to dom0 is that it takes time to re-purpose it went domUs are started. On a host with 128GB Ram you're getting things stuck for minutes while dom0 is ballooning down to make space.

Dom0 VCPUs

You might find that if you are running Xen on a large machine with more than 12 core, limiting the number of Dom0 vcpus to 8 and pinning them can improve performances. See: Can I dedicate a cpu core (or cores) only for dom0? .

HAP vs. shadow

HAP stands for hardware assisted paging and requires a CPU feature called EPT by Intel and RVI by AMD. It is used to manage the guest's mmu and it is usually faster than its fully software alternative (called shadow pagetabled). HAP can be enabled/disabled passing hap=0/1 in the VM config file.

PV vs PV on HVM

Linux, NetBSD, FreeBSD and Solaris can run as PV or PV on HVM guests (see this page and this page for more info). Memory intensive workloads that involve the continuous creation and destruction of page tables can perform better when run in a PV on HVM guest. Examples are kernbench and sql-bench. On the other hand memory workloads that run on a quasi-static set of page tables run better on a PV guests. An example of this kind of workloads is specjbb. See Xen_Linux_PV_on_HVM_drivers#Performance_Tradeoffs for more details.