Everyday Virtualization

Blog archive

Thin-Provisioning: Virtualization or Storage System?

Thin-provisioning, a great feature for virtual machines, uses the disk space required instead of what is allocated. I frequently refer to the Windows pie graph that displays used and free space as a good way to explain what thin-provisioned virtual disk files “cost” on disk. All major hypervisors support it, and there is somewhat of a debate as to whether or not the storage system or virtualization engine should manage thin-provisioning.

On the other hand, many storage systems in use for virtualization environment support a thin-provisioned volume. This is effectively aggregating the same benefit across many virtual machines. In the case of a volume that is presented to a physical server directly, thin-provisioned disks can be used in that way as well. Most mainstream storage products with decent management now support thin-provisioning of virtual machine storage types, primarily VMware's vStorage VMFS and Windows NTFS file systems. NTFS is probably marginally more supported simply due to Windows having broader support. A thin-provisioned volume on the storage system effectively is VMFS- or NTFS-aware for what is going on inside. This means that it knows if a .VMDK file is using its full allocation.

The prevailing thought is to have the storage system perform thin-provisioning for virtualization volumes. There is an non-quantified amount of overhead that may be associated with a write operation that would extend the file. You can hedge this off by using the 8 MB block size on your VMFS volumes, due to its built-in efficiencies or allocating a higher performance tier of disk. For the storage system to manage thin-provisioning, you're using on the disk controllers to manage that dynamic growth with direct access to write cache. What probably makes the least sense is to thin-provision on both environments. The disk savings would be minimal, yet overhead would be increased.

How do you approach thin-provisioning? Share your comments below.

Posted by Rick Vanover on 01/27/2010 at 4:32 PM


Reader Comments:

Sat, Jan 30, 2010 GregM Long Beach, CA

True Thin Provisioning (TP) depends on the capabilities of the SAN and the technologies enabled by the SAN vendor and/or other parties. I have talked and worked with a number of SANs and SAN vendors (EMC, Hitachi, NetApp, 3Par, HP, Dell Left-Hand, etc...). To my knowledge, Compellent is the only vendor who’s managed to TP Windows NTFS using a server-side agent. The agent communicates back with the SAN, which simplifies the management of actual used/available volume space at the SAN level. Currently, to my knowledge their no support for VMDKs from any vendor. It would most likely require VMware to support the capabilities in order for SAN vendors to take advantage of the technology; like Compellent has with NTFS. FYI… I am sure there's more involved from a technical stand point to enable this ability.
As with Compellent’s implementation the agent (support) needs to be on the server/host side. And, then VMDK TP can be implemented at the SAN level by SAN vendors to better enable the managibility of used/available SAN storage on both an a Tier and LUN perspective.

Fri, Jan 29, 2010 Stefan Holzwarth Germany

In my opinion both TP mechanism are needed. The central question is: How much reserve will you keep on your vmfs for your growing thin provisioned vm's? Boths opportunities have their trade off: risk or a lot of managment and unused, allocated storage If you deploy large, vmfs volumes that are thin provisioned at the storage artray you can handle your storage needs very efficient. Setup a policy and use only e.g. 50% of vmfs capacity and keep the rest for unpredictable vm growth.

Thu, Jan 28, 2010 nate

If your using a shared array use array based TP, if your using a dedicated array then use VM-based TP. To me the reason for this is if you have a shared array and say give your VMFS a 500GB volume that is not TP, that space has been consumed on the array, and cannot be used for anything else. VMware may be able to TP it itself, but that doesn't mean it takes less than 500GB on the array. You could of course start out with a small volume on the array and potentially grow it while taking advantage of vmware's TP, but that adds quite a bit of work for monitoring and stuff, and isn't automatic for sure. As for using both, the only benefit I can see if you being able to get a bit more insight into how much real space each VM is actually using from vCenter itself. Though it is annoying in that the space doesn't seem to be adjusted in real time, you have to click to refresh it(as far as I can tell). At the same time the results that vCenter gives could be wrong at the volume level, so be sure to monitor it at the array level in any case. Now if your array *only* does VMware stuff then you may not need to concern yourself as much, but my arrays always do more than just one thing.

Thu, Jan 28, 2010 Rick Vanover Grand Rapids, MI

Eric: Thanks. In the case of Chad's recommendation, that makes sense. I wonder, however, for other storage products. Ideally every storage product would have a recommendation on the practice.

I am curious, however, what the net storage gain would be from doing both. I'd have to imagine it to be a minimal (sub 5%) difference by double-dipping on thin provisioning.

Wed, Jan 27, 2010 Eric Siebert

I've heard to use both if you can, Chad Sakac is one of the persons who has said this and did a whole post on it: http://virtualgeek.typepad.com/virtual_geek/2009/04/thin-on-thin-where-should-you-do-thin-provisioning-vsphere-40-or-array-level.html It does become more complicated to monitor though and personally I might only do one instead of both.

Add Your Comment:

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above