vStorage VMFS Locking, Blocking Details

In case you didn't notice, I am a big fan of VMware vStorage VMFS. I recently pointed out some notes about slight version differences with vSphere's incremental update to the clustered file system for VMs. VMFS has other features of interest to me, like the coordinator-less architecture for file locking and smart blocking.

For many other clustered file system solutions, file lock management can be IP-based. VMFS, by its architecture, has all file locks performed on the disk with a distributed lock manager. Stepping back and thinking about that, it may sound like a bad idea at first -- that may not be a good idea on a disk volume for a large number of small files. VMFS volumes contain a small number of very large files (virtual disks). This makes the on-disk overhead of managing the locks negligible from the storage side, but very flexible from the management side. Further, when the lock management is being done on-disk, this takes away the possibility of split-brain scenarios where locking coordination can be interrupted or contended.

Comparatively, this is one of my sticking points of why I like VMFS storage compared to a Microsoft virtualization solution that is reliant on Microsoft Clustering Services. Hyper-V uses MSCS to create a clustered service for each logical unit number assigned to the host. Within each LUN, the current practice is to assign one VM to each LUN. Windows Server 2008 R2 introduces clustered shared volumes, yet still relies on MSCS. MSCS isn't available for the free Hyper-V offering, as MSCS requires Windows Server 2008 Enterprise or higher.

Because VMFS is coordinator-less, it has no requirement for MSCS or vCenter Server to manage the access. For this reason, administrators can move VMs between managed and unmanaged (ESXi free) environments with ease.

Besides the file system being smart on locking, it is also smart on block sizes. When a VMFS volume is formatted, the allocation unit size of 1 MB, 2 MB, 4 MB or 8 MB is selected. This permits virtual disk files (VMDKs) to be 256 GB, 512 GB, 1 TB or 2 TB, respectively. VMFS sneaks back into cool by provisioning a sub-block on the file system for the smaller data elements of a VMFS volume. On a traditional file system with an 8 MB allocation unit, a 2 KB file (like a .VMX file) would consume a full 8 MB. VMFS uses smaller sub-blocks to minimize internal defragmentation.

Is this too much information? Probably, but I can't get enough of it. Send me your comments on VMFS or leave a note below.

Posted by Rick Vanover on 06/10/2009 at 12:47 PM