NUMA node-specifc memory allocation
Entry point: xenguest --mode hvm_build
xenguest --mode hvm_build
:
It calls
do_hvm_build()
,
which calls
stub_xc_hvm_build()
.
stub_xc_hvm_build()
It starts the HVM/PVH domain creation by filling out the fields of struct flags
and struct xc_dom_image
and calls hvm_build_setup_mem()
.
hvm_build_setup_mem()
- Gets
struct xc_dom_image *dom
,max_mem_mib
, andmax_start_mib
. - Calculates start and size of most parts of the domain’s memory maps
- taking memory holes for I/O into account, e.g.
mmio_size
andmmio_start
.
- taking memory holes for I/O into account, e.g.
- It then uses those to calculate
lowmem_end
andhighmem_end
. - Finally, calls
xc_dom_boot_mem_init()
.
xc_dom_boot_mem_init()
In all cases, xc_dom_boot_mem_init()
is called.
It calls the architecture-specific meminit
hook for the domain type:
rc = dom->arch_hooks->meminit(dom);
meminit_hvm()
, the x86 HVM meminit() to allocate HVM domain memory.
It is the meminit
hook for x86 HVM domains:
https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1348
Dynamic memory / Populate on Demand
When dom->target_pages
is smaller than dom->total_pages
,
the X86 meminit_hvm()
function enables XENMEMF_populate_on_demand
:
https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1368
In this case, meminit_hvm()
function calls xc_domain_set_pod_target()
to set the populate on demand target to dom->target_pages
:
https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1454
meminit_hvm
: vmemranges
The allocation of domain boot memory is defined using vmemranges
.
meminit_hvm()
creates a default vmemranges
array based on the amount of
lowmem
(below 4G, untildom->lowmem_end
) andhighmem
(above 4G, untildom->highmem_end
).
The default vmemranges
do not carry information about from which NUMA node the memory shall be allocated.
meminit_hvm()
uses two vmemranges by default:
0
todom->lowmem_end
4G
todom->highmem_end
The NUMA node IDs (nid
) are set to 0:
- A dummy
vnode_to_pnode[]
fornid
maps0
toXC_NUMA_NO_NODE
.
Code: https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1371
Alternatively, callers of the meminit
hook have the possibility to pass
an array of memory ranges in dom->vmemranges
,
which can contain a specific NUMA node for each vmemrange
andthe "exact
" flag:
While vmemranges
have been added for vNUMA, meminit_hvm()
does not differentiate on that.
It is only concerned about memory init.
Hence, its code always acts on vmemranges
to avoid code duplications
and creates default non-NUMA vmemranges
.
If the caller does not pass specific dom->vmemranges
for NUMA cases, the default vmemranges
are used.
Caveat: When passing dom->vmemranges
, Populate-On-Demand (target_pages
< total_pages
) cannot be used:
With custom vmemranges
as Populate-On-Demand does not support NUMA.
As a result, memory overcommitment with ballooning is currently impossible for NUMA node-specific memory allocations.
Memory allocation of the Domain’s memory
meminit_hvm()
attempts to allocate 1GB pages if possible, falls back on 2MB pages if 1GB allocation fails,
and 4KB pages will be used eventually if both fail:
https://github.com/xen-project/xen/blob/master/tools/libs/guest/xg_dom_x86.c#L1475
For each vmemrange
, new_memflags
are composed, based on the base memflags
:
unsigned int new_memflags = memflags;
unsigned int vnode = vmemranges[vmemid].nid;
// With custom vmemranges, vnode_to_pnode is custom too:
unsigned int pnode = vnode_to_pnode[vnode]
if ( pnode != XC_NUMA_NO_NODE ) // vnode maps to a physical NUMA node:
// XENMEMF_exact_node: (XENMEMF_node(n) | XENMEMF_exact_node_request)
new_memflags |= XENMEMF_exact_node(pnode);
Thus, to allocate memory in a specific NUMA node, this is needed:
- Populate-On-Demand (POD) must not be used (
target_pages == total_pages
) - The sum of the
vmemranges
equalsdom->total_pages
dom->vmemranges
must be passed with anid
dom->vnode_to_pnode
must be passed, thenid
must map to the pNUMA node to allocate on.dom->nr_vmemranges
anddom->nr_vnodes
must be set accordingly
For the allocation of each vmemrange
(extent), a call to xc_domain_populate_physmap()
is used.
It calls the XENMEM_populate_physmap
hypercall for each group of extents to allocate.
Other callers of the meminit()
calls
libxl
calls xc_dom_boot_mem_init()
using libxl__build_dom()
from init-xenstore-domain.c/build().
Populating a domain’s “physical” memory
Hypercall: XENMEM_populate_physmap
XENMEM Call entry point into the Hypervisor: memory_op()
The entry point for all XENMEM
phypercalls is the function memory_op()
:
https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1395
memory_op()
checks the hypercall command: If the command is
- XENMEM_increase_reservation,
- XENMEM_decrease_reservation, or
- XENMEM_populate_physmap,
it copies the arguments from the calling guest and calls construct_memop_from_reservation()
.
Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1424
If it returns true, depending on the hypercall command, it calls
increase_reservation()
,decrease_reservation()
, orpopulate_physmap
using the struct memop_args
populated from the given xen_memory_reservation
.
construct_memop_from_reservation()
Populates struct memop_args
using the xen_memory_reservation
that the calling domain passed to one of the called hypercars.
https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1022
construct_memop_from_reservation()
- Copies the passed parameters from
xen_memory_reservation
:extent_start
nr_extents
extent_order
mem_flags
, and
- Converts
xen_memory_reservation->mem_flags->address_bits
toMEMF_bits
set inmemop_args->memflags
- If a
vnode
was passed, forvnuma
and if enabled, it passes thepnode
:XENMEMF_vnode
is passed inxen_memory_reservation->mem_flags
,memop_args->domain->vnuma
is set, andmemop_args->domain->vnuma->nr_vnodes
is !=0
, it gets thevnode
fromxen_memory_reservation->mem_flags
. Whenmemop_args->domain->vnode_to_pnode[vnode]
is notNUMA_NO_NODE
,- it:
- converts the
pnode
tomemop_args->memflags
- converts a
XENMEMF_exact_node_request
tomemop_args->memflags
- converts the
- If
XENMEMF_vnode
was not set in the passedxen_memory_reservation
,
it calls propagate_node()
to convert mem_flags
to memflags
.
Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1048
propagate_node()
- If the node in
xen_memory_reservation->mem_flags
isNUMA_NO_NODE
,
it returns true
(there is nothing to propagate)
- If the domain is running the function is Dom0 or a privileged domain:
- If the node is >=
MAX_NUMNODES
, returnfalse
- Else, convert the
node
and XENMEMF_exact_node_request tomemflags
- If the node is >=
- Otherwise, if
XENMEMF_exact_node_request
is set, returnfalse
. This means, to propagate theXENMEMF_exact_node_request
and the node, the propagation fails. - Otherwise, return
true
(noXENMEMF_exact_node_request
propagate)
Code: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L524
memory_op()
then calls populate_physmap()
to populate the memory
memory_op()
calls populate_physmap()
here: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L1458 -> https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L159
populate_physmap()
populate_physmap()
loops over the extents in the reservation it shall populate: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L197
For each extents in the reservation, it calls alloc_domheap_pages(d, a->extent_order, a->memflags)
: https://github.com/xen-project/xen/blob/master/xen/common/memory.c#L275
Function in xen/common/page_alloc.c
: alloc_domheap_pages()
alloc_domheap_pages() calls alloc_heap_pages().
Function in xen/common/page_alloc.c
: alloc_heap_pages()
alloc_heap_pages() calls get_free_buddy()
Description of get_free_buddy()
(in xen/common/page_alloc.c
)>
get_free_buddy()
is the primary function of the Xen buddy allocator:
It tries to find the best NUMA node and memory zone to allocate if possible.
Input parameters
- Zones to allocate from (starts at
zone_hi
untilzone_lo
) - Page order (size of the page)
- populate_physmap() callers start with 1GB pages and fall back
- Domain struct
Allocation algorithm
- Its first attempt is to find a page that matches the page order on the requested NUMA node(s).
- If that does not check out, it looks to break higher orders, and if that fails too, it lowers the zone until
zone_lo
.
![]() |
Note: If the function splits larger pages to not wait for scrubbing,
this could lead to fragmented memory with many small pages. |
![]() |
Improvement to be investigated:
Thus, the code should be updated to wait for scrubbing on the NUMA node to be completed for the requested page order before allocating memory. |
If all fails, it
checks if other NUMA nodes shall be tried (line 933-955):
vNUMA functionality of get_free_buddy()
:
Intro: With vNUMA, specific memory ranges are mapped from specific NUMA nodes.
Thus, for vNUMA domains, the calling functions have to pass one specific NUMA node to allocate from, and they would also set MEMF_exact_node
.
If a NUMA node was specified in the passed memflags, allocate from it, if possible.
If MEMF_exact_node
was set in the given memflags
, it does not fall back to generic node affinities.
Otherwise, it falls back to the next fallback.
Fallback: Generic NUMA functionality
Code location: get_free_buddy()
:
For the generic NUMA affinity, the domain should have one or more NUMA nodes in its struct domain->node_affinity
field when this function is called.
If node affinities are set up for the domain, it tries to allocate from the NUMA nodes struct domain->node_affinity
field in a round-robin way using the next NUMA node after the previous NUMA node the domain allocated from.
Otherwise, the function falls back to the default fallback.
Default fallback: Generic round-robin allocation =
When the above did not apply (not NUMA-specific alloc, MEMF_exact_node
not set in memflags
), then:
- All remaining nodes are attempted in a round-robin way using the next NUMA node after the NUMA node of the previous NUMA node that the domain allocated memory.