Pure Performance: XenServer Roars Off the Line

We could have looked at many different hypervisors, but decided to limit our testing to three -- ESX from VMware, XenServer from Citrix, and Microsoft's Hyper-V. One of the reasons was space -- in a print magazine, you simply don't have the real estate to make a story as long as you want.

The other reason was that most customers trying these things out will most likely be looking at one of these three. ESX is obvious -- it's numero uno when most admins start kicking hypervisor tires. Microsoft has put a lot of its marketing power behind Hyper-V, and it's quickly gaining a lot of recognition in the industry. XenServer, now that it's being given away with a lot of free enterprise-class features, is sure to go on most admins' short list.

The results, from writer/online columnist Rick Vanover, were startling, to say the least. The Porsche of hypervisors? XenServer. Raise your hand if you saw that coming. It outperformed Hyper-V and ESX in most categories. The pokiest? ESX. Again, not at all what I expected. In fact, even in the few tests ESX came out on top, it barely edged out the competition. Microsoft did well across the board, and is definitely a fine product.

One note about these results: First, as some of you know, VMware is very careful (some might say paranoid) about allowing benchmarking and performance results to be publicly released. Rick Vanover and I discussed our methodology with VMware, and the company agreed that our testing was fair. Note that we made no changes -- at all -- to our results, or the reporting of those results, due to our conversation with VMware. It's an important point, so you know that what we're reporting hasn't been altered in any way by any vendor.

So, what does this test tell you? One thing, and one thing only: how these specific hypervisors perform under these specific test conditions. That's it. Don't extrapolate that to mean we're saying XenServer is the "best" hypervisor, or ESX is the "worst"; that would be a mistake. As Rick points out, we didn't judge critical factors like management software, availability of third-party applications and so on. Finding the right hypervisor is a crucial decision for your shop, and a multitude of factors go into that decision. Speed is just one aspect; how important that is will depend on your virtualization goals.

What this does tell you is that from a pure performance standpoint, XenServer gets you there from here the fastest.

You'll notice that we also did a separate test with ESX, using memory overcommit to determine the effect on performance. This was for informational purposes only. Memory overcommit, available only with ESX, is quite important in many virtualization implementations, so we felt it would be valuable to show you the performance hit on your datacenter while it's in use.

Keep in mind that Rick, the author, is a confirmed VMware man, and uses ESX extensively on his production network. That makes it harder for those who want to scream "anti-VMware bias!" at us. As always, we call 'em as we see 'em here at Virtualization Review.

Do these results give you pause? Do they make you want to try XenServer or Hyper-V, when you wouldn't have previously? E-mail me or respond in the comments section below.

Posted by Keith Ward on 03/09/2009 at 12:48 PM