I was meeting with a new customer right before the holidays and part of the conversation was that they were very unhappy with the performance of their blade system in a virtualized environment. They told me that they benchmarked the same workload on rack-mount servers versus blades and they saw the CPU Ready Time drop significantly using the rack-mounted servers. They asked if I I had seen that with other customers because, they were considering moving away from blades. This was a large customer with over 2,000 VMs in production.
A week later, in an amazing coincidence, I had another customer call and complain about the exact same thing, except this time the customer said that the workload would vary depending on the blade the VM was on. They were seeing this performance degradation on their VMware View environment running Windows XP with a single vCPU.
I recalled reading something of the sort on the VMware communities' forum about processor power management causing VMs to run sluggish, so naturally we checked that first. In both cases the processor ower management was set to Dynamic at the blade BIOS level. Keep in mind that processor power management can be managed in two location: in ESX or in the hardware. It's also worth mentioning that the only time ESX can actually manipulate the processor power management settings is if the hardware configuration is set to "allow OS-controlled power management." During our inspection we found that it was not the case, so we started looking at the hardware. There, we found that it was set to Dynamic. As soon as we disabled that setting, the environment started to function perfectly again.
The processor power management setting affected mostly workloads that were CPU latency sensitive such as a VDI or XenApp/Terminal Server environment. However, I truly recommend that you disable this feature unless there is a compelling reason to use it. VMware also published a knowledge base article on this issue found here.
If you have experienced this, drop a note in the comments section so that we can help others.
Posted by Elias Khnaser on 01/09/2012 at 12:49 PM
More Tech Library