Tips

Is it Ever OK to Overcommit Host Memory?

Although it should be discouraged, there are times when it may have to be done.

• By Brien Posey
• 02/14/2017

Although hypervisors such as VMware ESXi and Microsoft Hyper-V make it possible to overcommit host memory, many virtualization admins avoid the practice in an effort to avoid the consequences that may come from running out of memory. But is memory overcommitment ever OK?

There are arguments both in favor and against memory overcommitment. Personally, I'm not a fan of the practice. As hosts run low on memory, a variety of performance and stability issues can occur. However, sometimes budgetary and operational requirements may mandate overcommitting memory. Memory overcommitment can be done safely, as long as you don't push things too far.

In a Hyper-V environment, an administrator can overcommit memory by way of the Dynamic Memory feature. Dynamic Memory allows a VM to claim physical memory when it is needed, and release that memory when no longer being used. An administrator can, of course, put thresholds in place to keep a VM from consuming too much or too little memory. There are also certain workloads (such as some database servers) that should not be used with dynamic memory.

In Hyper-V, an administrator can configure a VM with a minimum memory value that is lower than its startup memory value. The idea is that some VMs consume extra memory as a result of the boot process, but no longer need that memory once the boot process has completed. Hence, it becomes possible to configure memory in a way that leaves the host with enough RAM to continue running all of the VMs, but not enough memory to reboot any of them, since the boot process would require extra memory.

Microsoft avoids this problem through the use of a smart paging file. This is essentially storage space  treated as additional RAM to allow VMs to boot in case the host memory has been overcommitted.

VMware does things a little bit differently. While some might argue that Hyper-V doesn't support "true" memory overcommitment, VMware actually lets VMs consume more virtual memory than is physically installed in the host. This is possible because of several memory management techniques that VMware uses.

One of these techniques involves transferring memory between VMs on an as-needed basis. This process works very similarly to Hyper-V's dynamic memory feature.

VMware also uses something called "memory sharing". The idea is that a VM may run multiple instances of a process, and may therefore contain duplicate memory pages. VMware can eliminate some of these duplicate pages through memory sharing, which is sometimes called transparent page sharing. It's worth noting that by default, this occurs only within VMs, not across VMs, as page sharing across VMs could introduce security risks.

One more thing VMware does to accommodate memory overcommitment is to enable memory compression. Like storage compression, memory compression uses data reduction techniques to shrink memory pages, placing pages that can be compressed to 2 KB or less into a compression cache.