Enabling Transcendent Memory (Linux v5.2 tops)
Update: TMEM has been removed. The latest upstream branch containing it seems to be s 5.2.x namely release v5.2.21.
Tested with Xen version 4.11.0 and Linux dom0 & domU 4.18.16.
TMEM enables dynamic memory allocation for the guests, and the shrinking is quite fast. In less that 3 minutes, it goes from 5GB to approx 900MB. And after quite a while, I even goes down to less than 700MB on idling guests. And this takes into account guests'
buff/cache that is still allowed to live, which saves lots of disk I/O hence increases performance, a lot.
Among others Linux kernel settings for the domU, provided by default with
xenconfig, you need to enabled those,
Since TMEM refuses to be built-in, but only as a module, you will have to deliver
/lib/modules/kernel-ver onto the PV guests anyways. And make sure
depmod -a gets executed at some point with the newly running kernel.
Enabling a swap space onto the guest is not mandatory. It works without swap. Also the additional
tmem kernel argument at boot time is not required. You just need to load the module once the guest is up and running, either manually,
or by enabling it at startup e.g. on Debian/Ubuntu,
echo tmem >> /etc/modules
You should see this from the system logs,
[ 37.471390] xen:tmem: frontswap enabled, RAM provided by Xen Transcendent Memory [ 37.471410] xen:tmem: cleancache enabled, RAM provided by Xen Transcendent Memory [ 37.471423] xen_selfballoon: Initializing Xen selfballooning driver [ 37.471425] xen_selfballoon: Initializing frontswap selfshrinking driver
And magic happens, its xentop MAXMEM(k) starts decreasing, and
free -m within the guest also reports decreasing memory.
Note. It seems that for TMEM, only the
memory= guest setting matters and that the
maxmem= setting actually disables it: it shows static values on xentop output.
This feature has been provided by Dan Magenheimer at Oracle.