NUMA node-specifc memory allocation

From Xen
Jump to navigationJump to search

Entry point: xenguest --mode hvm_build

xenguest --mode hvm_build: It calls do_hvm_build(), which calls stub_xc_hvm_build().

stub_xc_hvm_build()

It starts the HVM/PVH domain creation by filling out the fields of struct flags and struct xc_dom_image and calls hvm_build_setup_mem().

hvm_build_setup_mem()

  • Gets struct xc_dom_image *dom, max_mem_mib, and max_start_mib.
  • Calculates start and size of most parts of the domain’s memory maps
    • taking memory holes for I/O into account, e.g. mmio_size and mmio_start.
  • It then uses those to calculate lowmem_end and highmem_end.
  • Finally, calls xc_dom_boot_mem_init().

xc_dom_boot_mem_init()

In all cases, xc_dom_boot_mem_init() is called.

It calls the architecture-specific meminit hook for the domain type:

rc = dom->arch_hooks->meminit(dom);

meminit_hvm(), the x86 HVM meminit() to allocate HVM domain memory.

It is the meminit hook for x86 HVM domains: https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1348

Dynamic memory / Populate on Demand

When dom->target_pages is smaller than dom->total_pages, the X86 meminit_hvm() function enables XENMEMF_populate_on_demand: https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1368

In this case, meminit_hvm() function calls xc_domain_set_pod_target() to set the populate on demand target to dom->target_pages: https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1454

meminit_hvm: vmemranges

The allocation of domain boot memory is defined using vmemranges.

meminit_hvm() creates a default vmemranges array based on the amount of

  • lowmem (below 4G, until dom->lowmem_end) and
  • highmem (above 4G, until dom->highmem_end).

The default vmemranges do not carry information about from which NUMA node the memory shall be allocated.

meminit_hvm() uses two vmemranges by default:

  • 0 to dom->lowmem_end
  • 4G to dom->highmem_end

The NUMA node IDs (nid) are set to 0:

  • A dummy vnode_to_pnode[] for nid maps 0 to XC_NUMA_NO_NODE.

Code: https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1371

Alternatively, callers of the meminit hook have the possibility to pass an array of memory ranges in dom->vmemranges, which can contain a specific NUMA node for each vmemrange andthe "exact" flag:

While vmemranges have been added for vNUMA, meminit_hvm() does not differentiate on that. It is only concerned about memory init. Hence, its code always acts on vmemranges to avoid code duplications and creates default non-NUMA vmemranges. If the caller does not pass specific dom->vmemranges for NUMA cases, the default vmemranges are used.

Caveat: When passing dom->vmemranges, Populate-On-Demand (target_pages < total_pages) cannot be used: With custom vmemranges as Populate-On-Demand does not support NUMA. As a result, memory overcommitment with ballooning is currently impossible for NUMA node-specific memory allocations.

Memory allocation of the Domain’s memory

meminit_hvm() attempts to allocate 1GB pages if possible, falls back on 2MB pages if 1GB allocation fails, and 4KB pages will be used eventually if both fail:

https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1475

For each vmemrange, new_memflags are composed, based on the base memflags:

unsigned int new_memflags = memflags;
unsigned int vnode = vmemranges[vmemid].nid;
// With custom vmemranges, vnode_to_pnode is custom too:
unsigned int pnode = vnode_to_pnode[vnode]

if ( pnode != XC_NUMA_NO_NODE )  // vnode maps to a physical NUMA node:
    // XENMEMF_exact_node: (XENMEMF_node(n) | XENMEMF_exact_node_request)
    new_memflags |= XENMEMF_exact_node(pnode);

Thus, to allocate memory in a specific NUMA node, this is needed:

  • Populate-On-Demand (POD) must not be used (target_pages == total_pages)
  • The sum of the vmemranges equals dom->total_pages
  • dom->vmemranges must be passed with a nid
  • dom->vnode_to_pnode must be passed, the nid must map to the pNUMA node to allocate on.
  • dom->nr_vmemranges and dom->nr_vnodes must be set accordingly

For the allocation of each vmemrange (extent), a call to xc_domain_populate_physmap() is used.

It calls the XENMEM_populate_physmap hypercall for each group of extents to allocate.

Other callers of the meminit() calls

libxl calls xc_dom_boot_mem_init() using libxl__build_dom() from init-xenstore-domain.c/build().

Populating a domain’s “physical” memory

Hypercall: XENMEM_populate_physmap

XENMEM Call entry point into the Hypervisor: memory_op()

The entry point for all XENMEM phypercalls is the function memory_op(): https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1395

memory_op() checks the hypercall command: If the command is

  • XENMEM_increase_reservation,
  • XENMEM_decrease_reservation, or
  • XENMEM_populate_physmap,

it copies the arguments from the calling guest and calls construct_memop_from_reservation(). Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1424

If it returns true, depending on the hypercall command, it calls

  • increase_reservation(),
  • decrease_reservation(), or
  • populate_physmap

using the struct memop_args populated from the given xen_memory_reservation.

construct_memop_from_reservation()

Populates struct memop_args using the xen_memory_reservation that the calling domain passed to one of the called hypercars. https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1022

construct_memop_from_reservation()

  • Copies the passed parameters from xen_memory_reservation:
    • extent_start
    • nr_extents
    • extent_order
    • mem_flags, and
  • Converts xen_memory_reservation->mem_flags->address_bits to MEMF_bits set in memop_args->memflags
  • If a vnode was passed, for vnuma and if enabled, it passes the pnode:
    • XENMEMF_vnode is passed in xen_memory_reservation->mem_flags,
    • memop_args->domain->vnuma is set, and
    • memop_args->domain->vnuma->nr_vnodes is != 0, it gets the vnode from xen_memory_reservation->mem_flags. When memop_args->domain->vnode_to_pnode[vnode] is not NUMA_NO_NODE,
    • it:
      • converts the pnode to memop_args->memflags
      • converts a XENMEMF_exact_node_request to memop_args->memflags
  • If XENMEMF_vnode was not set in the passed xen_memory_reservation,

it calls propagate_node() to convert mem_flags to memflags.

Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1048

propagate_node()

  • If the node in xen_memory_reservation->mem_flags is NUMA_NO_NODE,

it returns true (there is nothing to propagate)

  • If the domain is running the function is Dom0 or a privileged domain:
    • If the node is >= MAX_NUMNODES, return false
    • Else, convert the node and XENMEMF_exact_node_request to memflags
  • Otherwise, if XENMEMF_exact_node_request is set, return false. This means, to propagate the XENMEMF_exact_node_request and the node, the propagation fails.
  • Otherwise, return true (no XENMEMF_exact_node_request propagate)

Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L524

memory_op() then calls populate_physmap() to populate the memory

memory_op() calls populate_physmap() here: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1458 -> https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L159

populate_physmap()

populate_physmap() loops over the extents in the reservation it shall populate: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L197

For each extents in the reservation, it calls alloc_domheap_pages(d, a->extent_order, a->memflags): https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L275

Function in xen/common/page_alloc.c: alloc_domheap_pages()

alloc_domheap_pages() calls alloc_heap_pages().

Function in xen/common/page_alloc.c: alloc_heap_pages()

alloc_heap_pages() calls get_free_buddy()

Description of get_free_buddy() (in xen/common/page_alloc.c)>

get_free_buddy() is the primary function of the Xen buddy allocator:
It tries to find the best NUMA node and memory zone to allocate if possible.

Input parameters
  • Zones to allocate from (starts at zone_hi until zone_lo)
  • Page order (size of the page)
  • populate_physmap() callers start with 1GB pages and fall back
  • Domain struct
Allocation algorithm
  1. Its first attempt is to find a page that matches the page order on the requested NUMA node(s).
  2. If that does not check out, it looks to break higher orders, and if that fails too, it lowers the zone until zone_lo.
Icon Info.png The buddy allocator does not attempt to use not yet scrubbed pages!

But, when memflags tells it MEMF_no_scrub, it uses check_and_stop_scrub(pg) on 4k pages to prevent breaking higher-order pages instead.

Icon Ambox.png Note: If the function splits larger pages to not wait for scrubbing,

this could lead to fragmented memory with many small pages.

Icon Info.png Improvement to be investigated:

Thus, the code should be updated to wait for scrubbing on the NUMA node to be completed for the requested page order before allocating memory.


If all fails, it checks if other NUMA nodes shall be tried (line 933-955):

vNUMA functionality of get_free_buddy():

Intro: With vNUMA, specific memory ranges are mapped from specific NUMA nodes. Thus, for vNUMA domains, the calling functions have to pass one specific NUMA node to allocate from, and they would also set MEMF_exact_node.

If a NUMA node was specified in the passed memflags, allocate from it, if possible.

If MEMF_exact_node was set in the given memflags, it does not fall back to generic node affinities. Otherwise, it falls back to the next fallback.

Fallback: Generic NUMA functionality

Code location: get_free_buddy():

For the generic NUMA affinity, the domain should have one or more NUMA nodes in its struct domain->node_affinity field when this function is called.

If node affinities are set up for the domain, it tries to allocate from the NUMA nodes struct domain->node_affinity field in a round-robin way using the next NUMA node after the previous NUMA node the domain allocated from.

Otherwise, the function falls back to the default fallback.

Default fallback: Generic round-robin allocation =

When the above did not apply (not NUMA-specific alloc, MEMF_exact_node not set in memflags), then:

  • All remaining nodes are attempted in a round-robin way using the next NUMA node after the NUMA node of the previous NUMA node that the domain allocated memory.