How-To

Hyper-V vs. vSphere: Which One's Better for Your Private Cloud?

Microsoft and VMware are in pitched battle in the private cloud space. Both are worthwhile contenders, so how do you pick the right one for your project?

• By Paul Schnackenburg
• 07/15/2013

(Note: While this article is focused on Hyper-V and vSphere, there are of course other very capable hypervisors such as Citrix XenServer, which was recently released totally as an open source project, KVM and others. Both Microsoft and VMware seem to see each other as the main rivals and are also the only ones that offer complete management and monitoring solutions capable of building a private cloud solution.)

While most comparisons of Microsoft Windows Server 2012 Hyper-V and vSphere Enterprise Plus 5.1 start with an extensive table comparing CPU and memory support (showing that Hyper-V has higher limits in almost every area), I'd like to compare just a few different values: Hyper-V hosts can have 320 Logical Processors, (vSphere, 160); Hyper-V, 4TB of RAM (vSphere, 2TB); and clusters can have 64 hosts (32) and 8,000 VMs (4,000). VHDX disks can be 64TB whereas VMDK top out at 2TB.

These figures really don't matter to most businesses. When was the last time you bought a server with 16 physical processors, each of them with 10 cores and Hyper Threading and 4TB of memory? But they do indicate that both hypervisors have the scalability that could be needed in a few years. It's worth pointing out the artificial limitations in vSphere, where, for instance, the Standard SKU can only create VMs with up to 8 vCPUs and Enterprise Plus is required for 64 vCPUs, whereas the previous figures are true across all versions of Hyper-V, including the free Hyper-V Server 2012.

Fault tolerance (FT) is often brought up as a technology where Microsoft has no equivalent and that's for good reason. FT only makes sense in very specific scenarios and given the limitation of only one vCPU (one runaway process can hang the whole server), there are also several other restrictions (see bit.ly/M5vS6 for specific details). Furthermore, FT only protects against hardware failure -- if a software problem occurs, that problem will be dutifully replicated to other VMs. A decade ago, building reliable hardware platforms was difficult and the most common cause of server failure was hardware. Today, the situation is quite different, with software issues being blamed as the main culprit.

As for free hypervisors, the free vSphere 5.1 tops out at 32GB of host RAM (up from 8GB in vSphere 5.0) and doesn't support vMotion, whereas Hyper-V Server 2012 has the same scalability as in Windows Server Hyper-V (4TB of RAM, and 320 Logical Processors) as well as full support for clustering, Live Migration and Storage Live Migration. It can also run from a USB stick in a similar way to what you can do with the vSphere hypervisor.

Hypervisor support for different OSes inside VMs is often a comparison point and VMware's list (see bit.ly/s8U3Yb) is certainly longer than the one for Hyper-V. However, the main difference is in the use of the word "support." Where vSphere will support Windows 95 as a VM, Microsoft will not. Of course, Windows 95 will run fine in Hyper-V but Microsoft doesn't support it because the company no longer provides regular support for Windows 95 (call help desk and they'll tell you they won't be able to assist you). Maybe a better word that would cause less confusion would be "compatible."

The same goes for Linux distributions that Microsoft supports -- many have been tested on Hyper-V and for those that are supported, the Microsoft help desk will assist you. Microsoft has support agreements with the commercial vendors for each particular distribution. Because of the work Microsoft has done to include extensions in the Linux kernel, any modern distribution should work fine in Hyper-V. But if it's not on the list (see the list of supported distributions at bit.ly/Hmje6V), Microsoft support probably won't be able to help you out.

The vSphere hypervisor supports hot-add of processors and network interfaces to a running VM, something Hyper-V doesn't support. Be aware that the guest OS also needs to support these hardware devices suddenly showing up.
Finally, comparing hypervisor performance has been hindered for many years by VMware, discouraging independent bench marking. So, for an unbiased -- albeit a bit VDI-focused -- view, take a look at loginvsi.com.
In short, the performance difference between hypervisors on equivalent hardware is minimal for most workloads.

Memory
VM memory management is another philosophical difference between the two hypervisors: VMware sees each VM as a black box and uses techniques that don't involve the OS inside the VM. Microsoft lets the hypervisor communicate with the OS inside the VM for more efficient memory operations. This is true across both Windows Server 2008 (and later) inside a VM and Linux distributions. Also, the vSphere hypervisor only invokes its memory management techniques such as ballooning and hypervisor swapping when the host is starved; Hyper-V applies Dynamic Memory on a constant basis across all VMs on a host (that are configured to use it), minimizing the risk of the host ending up in a starved state.

Transparent Page Sharing (TPS) is often brought up as a reason for being able to achieve higher VM density on a given host compared to Hyper-V (for more on TPS, go to bit.ly/14x7iM0). But modern OSes use 2MB memory pages compared to 4KB pages in older ones, making the chance of finding identical pages slim. TPS also places a higher load on the host's processors to do the calculations and it takes some time to achieve good results, which may not work well if you have a datacenter where VMs move around frequently.

Storage
While Windows Server 2012 offers Storage Spaces and a built-in iSCSI target, there's no direct equivalent of the vSphere Storage Appliance (VSA). VSA is an interesting small business solution that transforms direct-attached storage (DAS) on two or three vSphere hypervisor hosts to highly available storage for VMs. Third-party vendors such as StarWind Software Inc. provide software iSCSI target solutions for easily building fault-tolerant storage on two Windows servers.

For enabling clustered hosts to communicate efficiently with storage-attached networks (SANs), VMware offers vSphere Storage APIs for Array Integration (VAAI), which is similar to offloaded data transfer (ODX) support in Windows Server 2012. Note that VAAI is a proprietary standard, but it enjoys widespread support because it's been around for quite a few years. Hyper-V has no equivalent to Storage DRS, which lets you control IOPS on a per-VM basis. System Center Virtual Machine Manager (VMM) can automatically provision a LUN on shared storage, whereas in vSphere the storage administrator has to create a LUN before it can be used by vCenter.

Shared storage in Windows Server 2012 that uses the Clustered Shared Volumes (CSV) file system can be protected by Bitlocker, there's no equivalent whole volume encryption in vSphere.

Networking
The Hyper-V network switch is extensible and there are already more extensions available than for vSphere (see my December 2012 article, "Deep Dive: What's New in Hyper-V" at VirtualizationReview.com/Schnack122012). It's important to note that the vSphere Enterprise Plus 5.1 (the only SKU to offer changes to the virtual network switch) switch isn't extensible, it's replaceable. The Cisco Nexus 1000v completely replaces the built-in switching functionality whereas extensions for the Hyper-V switch merely add value on top of the built-in functionality.

For network protection (ARP and ND Spoofing Protection, DHCP Snooping Protection and DHCP Guard, along with Virtual Port Access Control Lists) there's no built-in functionality in vSphere and this requires either the Cisco switch or the vCloud Networking & Security (vCNS) product (formerly vShield App) at additional cost. Network QoS is only available in the Enterprise Plus SKU, whereas Hyper-V offers it in every edition.

VMware has no support for using IPSec with vMotion, whereas Hyper-V has been tested extensively on IPSec because the internal development network as Microsoft has a requirement to always use IPSec.

Single Root I/O Virtualization (SR-IOV) is the future of virtualization/acceleration of NICs and is supported by VMware (known as DirectPath I/O) but not for vMotion, so you can't move a VM using this technology. This loss of VM mobility is a big compromise because one of the biggest benefits of server virtualization is being able to move VMs around. There are other limitations to DirectPath I/O, such as a short list of supported NICs and no support for Memory Overcommit, VM Snapshots or Suspend/Resume. It should be mentioned that some of these limitations are solved on specific models of the Cisco UCS platform. Microsoft supports SR-IOV across editions and lets you live migrate a VM using SR-IOV to a host without an SR-IOV NIC as well as to another host with an SR-IOV NIC.

For guest clustering, be aware that although both platforms support this, VMware has some limitations on the type of configurations that are supported (see bit.ly/fW0nkX), including the maximum number of nodes (two or five in 5.1), whereas Microsoft doesn't have these limitations (64 nodes).

Microsoft supports Dynamic VM Queue functionality (intelligently routing network traffic to VMs on a host) on any NIC where the driver supports DVMQ across 1Gb and 10Gb NICs, whereas VMware only supports this functionality on some 10Gb NICs.

Management and Monitoring
vCenter is often compared to the bare bones UI of Hyper-V manager in Windows Server 2012, but the true comparison should be between VMM and vCenter. The System Center suite, especially with the recent release, is a stepping stone to both the private and public cloud whilst still managing your existing physical and virtualization investment.

In vCenter each VM template creates a separate VMDK, whereas VMM can build many different deployments on top of a single Windows Server VM template. VMM can also enable roles and features of a Windows Server VM at deployment time, whereas VMware's solution requires a separate template for each type of VM (or scripting to enable the role or feature after deployment).

If you compare individual parts of System Center with their VMware counterparts, you'll notice there's a distinct focus on VMware's own products. This differs from System Center, as the suite is more open to other platforms. vCenter Orchestrator, for instance, is focused on VMware's own products, while System Center Orchestrator has plenty of integration packs to work with other platforms. The list includes HP iLO and HP Operations Manager, IBM Tivoli Netcool and vSphere along with community developed IPs for a variety of platforms.

For service management and private cloud self-service VMware offers DynamicOps but it isn't aligned to Information Technology Infrastructure Library (ITIL) processes, nor is it based on a single central configuration management database (CMDB) for all ITIL resources. The System Center equivalent, Service Manager, is built around a CMDB and ITIL processes.
In the VDI space Hyper-V can be used to host both server and client OSes and they're managed with the same tools. For

VMware, View 5.1 is a separate product with additional licensing costs.

For hybrid environments, VMM manages both 4.1 and 5.1 setups of vSphere with full support for VMware VMs to be incorporated into VMM services using vMotion and Dynamic Optimization. While the Multi-Hypervisor Manager plug-in for vCenter can manage Hyper-V, it can't live migrate VMs from host to host, and that severely blunts its usefulness.

VMware only backs up VMs as whole VMs -- there's no application awareness, and third-party backup tools are required for this. And while Site Recovery Manager (SRM) for disaster recovery is a good solution, it's definitely a big business option because of the associated configuration and infrastructure requirements. System Center Data Protection Manager (DPM), on the other hand, offers application-aware backups for both the physical and virtual world, plus built-in disaster recovery by replicating one DPM server to a second DPM server in another datacenter.

Not the Only Choice
If there's any doubt that VMware is looking at the newest version of Hyper-V as serious competition, consider the new features that VMware brought into vSphere 5.1 after the release of Hyper-V in Windows Server 2012. There's Enhanced vMotion (Shared Nothing Live Migration) and vSphere Replication (Hyper-V Replica). The latter can't be automated (unless you've already got SRM, which defeats the small business appeal) and you can't test failover, nor can you give VMs alternative IP addresses prior to a failover.

Ultimately, the discussion of technology advantages of one product over another isn't going to make a big difference, although it's important to know so you can base your decisions and recommendations on current facts. The bottom line is that VMware created a comprehensive virtualization platform and encouraged an impressive eco system around it, capturing a large portion of the enterprise market, and for quite some years had very little competition.

But the world is changing and if you're looking at building a private cloud for your business, VMware solutions are no longer the only choice.

Finally, it comes down to a difference in philosophy. For VMware, the workload inside the VM can be anything and its aim is to provide the best virtualization and private cloud platform at a price point that reflects a superior product. Microsoft sees server virtualization as simply an extension of the OS, and because most VM workloads are based on Windows, the company has no reason to make money out of the virtualization layer itself. And given Hyper-V is now as good as vSphere but a lot less expensive, it's only a matter of time before market forces will change the balance in favor of Hyper-V.

The world is also changing; the virtualization war isn't just fought in big-business datacenters. The public cloud is now a larger theater of operations and here is where VMware's approach is muddier than Microsoft's. With the recent announcement of its own Hybrid Cloud Service offering, VMware has a lot of catching up to do. In the end, if I want to host VMs in a public cloud I don't really care about the underlying hypervisor, I care about cost, uptime, security and ease of management. The catch, of course, is that it'll be easier to move a VM from my own vSphere infrastructure to the VMware Hybrid Cloud Service than to Amazon Web Services (AWS) or Windows Azure (just as it's simple to move a VM from VMM to Windows Azure using AppController). It's hard to see VMware competing with a premium label and pricing against established players such as AWS and Windows Azure. Microsoft has 19 years of experience in running large "cloud" services on the Internet; this experience is clearly making its way back into products such as System Center and Windows Server 2012, and VMware will have to prove it can muster the equivalent public cloud chops.

System Center 2012 is equally adept at managing and monitoring your physical and virtual IT assets; making the transition for many enterprises from physical servers to virtualization easy. It's also built for private cloud, easing the next step in the journey and even ties in with the public cloud to enable businesses to expand their IT infrastructure to the public cloud at their own pace. VMware's management and monitoring toolset is more fragmented, often having come from acquisitions. And although some monitoring of the physical world has been added in recent versions, it's still not as good as the deep monitoring and resolution suggestions that Operations Manager offers.

Do I doubt that VMware is going to match or exceed Microsoft in the areas it's now trailing (scalability and VM mobility), or that the company will adjust its pricing structure to compete with vSphere 5.2 or 6 or whatever the next iteration will be called? Of course not -- the point is that VMware is no longer alone on top of the hill and it'll now have to fight to get the crown back -- and that sort of competition will benefit all of us as customers.

Can't Get Enough Hyper-V?
This article is the end piece of my eight-part series deep dive into the newest features of the latest general release of Hyper-V.