Whether or not you use any form of hybrid cloud, I think it's important to at least know what's available, and how it's being used. There are two primary uses: One is to extend the traditional datacenter to the cloud, including virtual machines (VMs), networking and storage. The other main use is in building cloud-native applications.
The Microsoft Azure cloud offers a lot of options for both uses, and continues to add services, making it extremely useful and flexible. I've listed some of my favorites here, demonstrating that Azure is a lot more than just a bunch of Hyper-V VMs running Windows.
- SQL Server Stretch Database: This service, which is currently in preview, enables "bottomless" cloud storage for SQL Server 2016 databases, while still having it on-premises. This is a hybrid application for perhaps one of the most critical applications in any datacenter.
The fundamental idea with a SQL Server Stretch database is that hot and cold data can be placed in the right type of storage. This is an absolute game-changer for those familiar with their SQL Server data profiles at a table and database level. For example, a critical application may use one database, but typically only a few tables are actively used. There may be other tables (taking up a lot of storage) that would be a good candidate for a SQL Server Stretch Database.
- DocumentDB: You don't have to use SQL Server, though; Azure has a fully managed NoSQL implementation. NoSQL has similarities to relational database technologies SQL, but scales out well; it also won't lock you into relationships that may not work at huge scale.
- Azure Active Directory: A big concern organizations have with any technology service is authorization and access. Active Directory (AD) is the arguably the most common authentication framework in the world for business datacenters, so it was natural for Microsoft to extend it to Azure. What may surprise you, however, is that Azure AD can integrate with its on-premises namesake. This is a good control element when services, accounts and access needs to be added, changed or revoked.
- Nearly 400 Linux Virtual Machines: As I said earlier, Azure isn't just a bunch of Windows VMs. Some of the most interesting Linux VMs I found include Wordpress, the Tenable Nessus security scanner, Oracle Database 12.1 and Docker on Ubuntu Server. Of course, the traditional Linux distros like Debian, CentOS, Asianux, Red Hat and SUSE have images as well.
- Cognitive Services: This service is in preview, but I want to think big here. This is a significant innovation for future frameworks such as additional factor authentication, security surveillance and even creating metadata on images after they're created. One example is the Face API, offering services like face verification, face identification, face searching and more. Other services include Recommendation API, Speech API, Emotion API and more. Fascinating stuff!
Within the next year -- or maybe even sooner -- I'll do an updated version of this post, listing more Azure innovation.
Do you see a use for new cloud services from Azure? Are you the hybrid cloud seeker? Are cloud-native apps making their way into your environment? Share your comments below.
Posted by Rick Vanover on 05/06/2016 at 12:30 PM0 comments
It's standard operating procedure to check the VMware Compatibility Guide before doing anything significant, like purchasing equipment, upgrading a cluster, installing a new feature and so on. The compatibility guide is a great online research tool, and it's important to give it a look when it comes to ensuring proper support.
If you have a four-year-old storage system, for instance, it doesn't make sense to purchase new servers and vSphere 6, then run it on that infrastructure for another four or five years.
For this to work, of course, you also need to know exactly what you've got. But from a host perspective, it can be difficult to determine what you have vs. what's in the compatibility guide. Fortunately, there's a command line tool for that: esxcfg-info. This is especially helpful for a large number of hosts, as the same command can be run for each.
I recently found esxcfg-info when I stumbled across VMware KB TV. The videos have practical tips for a number of tasks, and one of the more helpful videos is titled "Confirming ESX/ESXi host hardware (System, Storage, and I/O) compatibility".
The esxcfg-info command is explained in the KB; if you run it with the recommended parameters (esxcfg-info | less –I), you get a long scroll onscreen scroll, as shown in Figure 1.
[Click on image for larger view.]
Figure 1. The esxcfg-info command reports on all things hardware for the ESXi host.
That's a lot of data! You'll eventually cancel out of this after the first few lines. If you have a number of hosts, you'll want to get this as a text file and analyze it on your computer, rather than on the SSH session of the Putty host.
I tweaked an esxcfg command to cause it to output to a file that is the named the system's hostname. Here's the command:
esxcfg-info > $(hostname).txt
The trick is to put it in a place that you can pick it up easily, which is why I've put it in a VMFS datastore, as you can see in Figure 2. Note that the file is around 9MB; it's bigger than you may think.
[Click on image for larger view.]
Figure 2. The file placed on a datastore becomes easier to centralize with the datastore browser.
This is one example where the default "datastore" name may be helpful in finding a file for locally attached storage.
I've found the esxcfg-info command to be a simple, effective way to reconcile VMware compatibility guide results with the contents of your ESXi host system.
Have you used esxcfg-info much? If so, how have you reviewed the results? Share your comments below.
Posted by Rick Vanover on 04/18/2016 at 12:17 PM0 comments
As virtualized infrastructures get more complicated, sometimes you need a way to troubleshoot things quickly, in simple terms above the infrastructure. One of the areas I've looked closely at is network throughput. This is partly because I've worked in a lot of virtualized infrastructures that use more NFS, SMB3 or iSCSI communication for VMs, and less fibre channel, as I've gone along.
While using the network storage protocols in lab environments in particular, I don't always separate interfaces that provide the storage network (iSCSI, NFS, SMB3 and so on) from the network interfaces that guest VM operating systems use. Whether in a lab or production setting, the iPerf tool is a handy way to measure network throughput above the infrastructure. The latest iteration is iPerf3.
If you're not familiar with iPerf3, it's time for an introduction. According to its Web home, iPerf3 is principally developed by ESnet and the Lawrence Berkeley National Laboratory, and released under a three-clause BSD license. I think it's the easiest way to do active measurements of network bandwidth between devices at the operating system level. The best part is that support, once limited to Windows and Linux, now includes Android, iPhone and MacOS X as well. (Note that the latest version, iPerf3, isn't backwards compatible with the first version).
I recommend putting this tool in your arsenal or downloading it as needed. It doesn't need to be installed (at least on Windows; I haven't used the Linux version of iPerf3, but note that some Linux distributions have it embedded as Iperf), and can function as a standalone tool. On a Windows system, I run the server side element as iperf –s
(this is shown in Figure 1).
[Click on image for larger view.]
Figure 1. iPerf3 clearly shows throughput and network communication.
Once that server process started, I went to another system and ran iPerf3 to connect to the first system. The –s parameter was run first to establish a server, then the –c parameter (and the host name),
to connect to it from a different system. It looks like this: like this: iperf3 –c d-think2. In Figure 1, you can see the connection established. Figure 2 shows the connection being made (highlighted in green).
[Click on image for larger view.]
Figure 2. Showing the connection being established.
Notice in Figure 2 that the second connection was much faster. This is because both the iPerf3 client and server roles are VMware VMs on the same port group. We'd expect to see a high throughput here as they use CPU resources for network transfer, rather than actively going over the network interface.
I also had a chance to install the very handy Android application to communicate to these VMs with iPerf3. While this is of course only accessible via Wi-Fi, it's an additional way to test the experience from the datacenter down to a device. The Android interface is shown in Figure 3.
[Click on image for larger view.]
Figure 3. iPerf3 (the capitalization is different in the PC and mobile device versions) on an Android device.
Practically speaking, this is very helpful for testing things like communication between VMs in different infrastructures. From a network role perspective, iPerf3 can also be helpful in an infrastructure that is very "stacked" on top of itself.
So learn iPerf3, and the next time someone says an application is slow, use this handy tool to look for clues, especially to and from other VMs that aren't reporting any issues.
Posted by Rick Vanover on 03/24/2016 at 2:26 PM0 comments
When Microsoft announced that the Microsoft Azure Stack would be a thing, I had to stop and think about the fundamental impact on the cloud. I'm a co-host of the In Tech We Trust podcast, and when this was announced on an episode, I latched on to it.
This is a very important time for IT practitioners. Everywhere I look, the cloud and service provider angle becomes an option in how datacenters are run, leveraging key technologies like virtualization and modern storage systems. The benefits of the cloud model are clear, with a crucial differentiator: the applications used. What I would call a cloud-stack application model vs. a traditional application-stack model is an important decision point for an enterprise.
However, the Azure Stack bridges an important gap here. The cloud style of consumption and user experience is provided, yet the application offerings are right in line with what IT practitioners have deployed and supported for years. Now, those offerings have been expanded with capabilities like containers.
Consider also the advantages of running Azure on your own infrastructure. I'm doing this now, and think you should, too. To that end, I've created a simple Q&A to help you get started:
What is the Azure Stack? It's a way to run the Azure experience on your own infrastructure. Azure Stack provides Infrastructure-as-a-Service (IaaS) and Platform-as-a-Service (PaaS) capabilities.
Does it only work with Microsoft OSes and applications? No. You'll see Linux images and third-party applications in the form of Azure Resource Manager templates.
Can I just throw it in to my environment as a VM? Not really. The Azure Stack has some significant requirements, and they should be taken seriously. Things in this document to note:
- 128 GB of RAM and 2 sockets with 16 cores recommended -- but not enforced
- Windows Server 2016 Datacenter Edition TP4 needs to be installed and updated
- A Microsoft Azure Active Directory account must be in place (Azure Active Directory costs $6 per user, per month)
Will I have to use PowerShell? Unfortunately for some, yes. A lot of the Azure Stack configuration elements in the preview are PowerShell scripts. The good news is that most of the scripts are provided, and that the finished product will be a Web experience that consumes the IaaS and PaaS resources.
I expect the Azure Stack to be a game-changer in the datacenter. It will answer many IT questions about its validity, and help determine if the cloud model will meet an organization's requirements.
Posted by Rick Vanover on 03/08/2016 at 9:19 AM0 comments
In vSphere 5.1, the vSphere tag framework was introduced, and has been improved steadily since. It's an organizational construct at a management level above the infrastructure, and is great for a number of things. Let's start by my own definition for each of these two concepts and then I'll outline some good use cases.
A tag category is a parent object of many tags, and can have a fundamental rule of permitting one tag per object or permitting many tags per object. This is called cardinality in the vSphere Web Client; for more, see the documentation.
A tag is a label applied to any object in vSphere (host, cluster, virtual machine, datastore and so on). This tag has a category assigned to it. Here's the VMware documentation reference for creating a tag.
[Click on image for larger view.]
Figure 1. An example tag category in the vSphere Web Client.
Possible Tag Uses
Now that you have a definition, the next step is to learn to use tags. Here are a number of use cases I think many environments can benefit from:
- Tag a GRC infrastructure. If there are any workloads subject to governance or risk management or compliance (GRC), this may be a good way to easily identify them in the Web Client. For example, one category could be called "In-Scope," and tags could be labeled "PCI", "HIPPA" or others; it's pretty clear what responsibility those VMs, datastores, hosts or clusters would be governed under.
- Use tags to define business and department ownership. If there are many stakeholders across the organization and possibly multiple sites, tags and categories can help identify role and ownership. An example would be categories such as "Business Systems" and tags such as "Accounting" and "Human Resources"; another category may be "Operations," with corresponding tags such as "Timekeeping" or "Manufacturing Support." This is of course in addition to (hopefully) names on VMs that make sense as well.
- Set up protection requirements. Tags and categories can be used to make a software-defined set of rules for things like backup, storage snapshots, offsite replication and more. Categories like "Backup" and tags like "Daily" or "12-Hour" can be used to design protection.
Here's a usage tip: Make your tags specific. That can include self-documenting tag and category names. "Category1" or "Test Tag" doesn't really tell us much about what the tag and category are being used for.
As you can see, there are a number of use cases here for the tag and category framework in vSphere. Have you been using this new mechanism yet? If so, what use cases have you found beneficial? Share your comments below.
Posted by Rick Vanover on 02/04/2016 at 9:58 AM0 comments
I find that the virtualization industry is very dynamic, yet I'm a bit guilty of having "tunnel vision" for the two most popular Type 1 hypervisors, VMware vSphere and Microsoft Hyper-V. But there are other good ones out there, and you should know about the options. To that end, here's a quick rundown on who's who for 2016 in the hypervisor market:
- VMware vSphere: The current version is vSphere 6.0 update 1 (b). Most are familiar with vSphere, as it's still the standard. A lot of innovation around this hypervisor has come in the storage industry, and more is coming.
- Microsoft Hyper-V: The current version of Windows Server is 2012 R2, which means the latest edition of Microsoft's hypervisor is Hyper-V Server 2012 R2 or Windows Server 2012 R2's Hyper-V role. Windows Server 2016 is right around the corner, with new Hyper-V features on the way.
- Huawei FusionSphere: I had no idea this was even an option! Currently on version 3.1, this is an implementation built on OpenStack, and is more relevant for cloud-stack data centers.
- Oracle VM: This hypervisor is on version 3.3 and has fans in large Oracle shops, as well as those looking for SPARC support in the datacenter. I used to love using Sun xVM VirtualBox; and even after the Oracle-Sun merger, it's still here (as a Type 2 hypervisor) and ready for download.
- Red Hat Enterprise Virtualization: This hypervisor is positioned to be an OpenStack implementation and is based on the KVM hypervisor. It's currently on version 3.5.
- KVM: Kernel Virtual Machine, or KVM, is on version 1.3 and is the pure standards-based hypervisor that Red Hat Enterprise Virtualization is based on. Many Linux distributions can add KVM (SUSE Linux Enterprise Server (SLES), for example).
- Citrix XenServer: This commercial hypervisor is based on the Linux Foundation Xen Project. Citrix's XenServer currently is on 6.5 SP1.
- IBM PowerVM: This is different from the others in that it's targeted for AIX and other enterprise datacenter systems. Further, it's limited to support on a few hardware systems (such as POWER6 and POWER7). Also in this category also is the IBM z/VM hypervisor -- currently on version 6.3 -- which provides a virtualization layer for mainframe systems. While not broadly applicable, it's interesting to note that the other types of enterprise data center platforms have a virtualization option.
Posted by Rick Vanover on 01/14/2016 at 9:46 AM0 comments
For the most part, I live in a world of default configurations. There are, however a number of small tweaks that I do to ESXi hosts after they have been installed to customize my configuration.
While this list is not specific-enough to work everywhere, chances are you will pick up something and say "Oh, yeah, that's a good idea!" I don't include specific networking or storage topics, as they apply to each specific environment, but this generic list may apply to you in each of your virtualization practices. Most if not all of these settings can be pushed through vSphere Host Profiles, but I don't always work in environments licensed to that level.
Here we go!
1. Set DNS.
Get DNS right. Get DNS right. Get DNS right. Specifically, I set each host to have the same DNS servers and suffixes. This also includes getting the hostname from localhost changed to the proper local FQDN. You are renaming your hosts from localhost, aren't you?
2. Create the DNS A record for the host.
Just to ensure your impeccable DNS configuration from step 1 is made right.
3. Set NTP time servers.
Once DNS is set correctly, I set the hosts to use this pool of NTP servers: 0.north-america.pool.ntp.org, 1.north-america.pool.ntp.org, 2.north-america.pool.ntp.org, 3.north-america.pool.ntp.org. This is set in the host configuration via the vSphere client (see Fig. 1).
 |
Figure 1. The NTP authoritative time server list is configured in the vSphere Client. (Click image to view larger version.) |
4. Raise the maximum number of NFS datastores.
I work in a world where we have multiple NFS datastores, including some that come and go. I did write an earlier blog post on this topic which you can read here.
5. Disable the SSH warning when it is enabled.
I'm sorry, sometimes I need to troubleshoot via the command line and I don't really want to get up from my desk. SSH does the trick. VMware KB 2007922 outlines how to suppress this warning in the vSphere Client.
6. Install correct licensing for the host.
Duh. This will get you in 90 days if you don't do it now.
7. Set any VM's for automatic startup.
This is an important feature when there is a problem such as a power outage. I recommend putting virtualized domain controllers and network infrastructure (DHCP/DNS) first, then critical applications but not necessarily everything else.
8. Set enable vMotion on vmkernel.
Not sure why this isn't a default. Of course environments with multiple vmkernel interfaces will need to granularly direct traffic.
9. Rename local datastores.
Nobody likes an environment littered with datastore, datastore (1), datastore (2), etc. I recommend a nomenclature that self-documents the datastore. I have been using this format: DAS-HostName-LUN1. DAS stands for direct attached storage, HostName is the shortname (not localhost!) for the ESXi system, and LUN1 would be the first (or whichever subsequent) volume on the local system. If you don't want to use direct attached storage, you can delete the datastore as well; the ESXi hosts partitions will remain. If you are using ESX 4.1 or earlier, don't do this however.
10. Test your remote access options.
This can include SSH, a hardware appliance such as KVM, Dell DRAC, HP iLO or others. You can't rely on something you can't support.
That is my short list of tips and tricks for post-installation tips and tricks for generic ESXi host installations. Do you have any post-install tips? Share your tips in the comments section!
Posted by Rick Vanover on 12/19/2012 at 12:48 PM3 comments
One of my absolute favorite features of Windows Server 2012 with Hyper-V is the VHDX disk file format. This new virtual disk format goes up to 64 TB and is the answer to many storage challenges for VMs when larger workloads end up as a VM.
While my virtualization practice frequently sees me leverage thin-provisioning, I don't always make virtual disk files exactly to the maximum size they can be made. There isn't too much harm in that point, unless the guest operating systems, of course, fill up. Monitoring discussions aside, there are situations where the VM will need its storage allocation increased.
Now, the one bummer here is that, with Hyper-V on Windows Server 2012 the virtual disk cannot do a dynamic grow while the VM is running. To answer your next question, we can't add a virtual disk either while the VM is running to the first IDE controller.
But you can add virtual disks to SCSI controllers when the VM is running. The VM does have to be powered off (not paused) to have a VHDX (or VHD) file expanded. Once the VM is shut down, edit the VM settings by right-clicking on the VM in Hyper-V Manager. Once you are in the VM settings, find the virtual disk on the VM (see Fig. 1).
 |
Figure 1. The VHDX can be edited when the VM is powered off to increase its size. (Click image to view larger version.) |
Once the edit option is selected for the VHDX file, four options come up. One of them, surprisingly, is to shrink the VHDX which is quite handy if you need it. Compacting the disk will reclaim space previously used by the guest VM, and the convert option will move it to a new VHD or VHDX with optional format changes. Select the Expand option to increase the size of a VHDX file (see Fig. 2).
 |
Figure 2. Expanding the VHDX file is one of the four options that can be done when the VM is offline. (Click image to view larger version.) |
The 64 TB maximum is a great ceiling for a virtual disk size. Given enough time, this surely will become too small, though I can't imagine it for a while. To increase the disk, simply enter the new number. It is important to note that if the disk is a dynamically expanding disk, it will not write out the new differenced amount, and the expand task will be quick. If the disk is of a fixed size, the expand task will require the I/O to the storage system to write the disk out. If the storage product is supported for Offloaded Data Transfer (ODX), the task may be accelerated by help from the storage array.
This is an easy task, but it currently has to be done with the VM powered off. How do you manage expanding disk sizes for virtual disks, further, what is your default allocation? I usually provide VMs 100 GB of space unless I know more going in. Share some of your practice points in the comments.
Posted by Rick Vanover on 12/13/2012 at 12:48 PM5 comments
There is no shortage of storage solutions today that position storage around some form of cloud technology. Love or hate the cloud buzzword, here is something different: Oxygen Cloud.
I got to check out the Oxygen Cloud technology and here are my thoughts. First of all, it's not what you think. In fact, I was struggling to say "what" the Oxygen Cloud is. At first it may just look like another file distribution tool, like DropBox. A closer look and I found that it's got a lot more going on.
The underlying technology is a virtual file system and that's the key to it all. It is the basis for the synchronization, and then a number of management and policy elements kick in. So, in a way it is the best of both worlds. Let's dig into the details a bit. Fig. 1 shows how the components work.
 |
Figure 1. The Oxygen cloud takes a fresh approach in leveraging on-premise storage and authentication. (Click image to view larger version.) |
The critical element here is that the storage delivered to the endpoints -- Windows, Mac OS, iOS, Android and HTTPS -- is an instance of the virtual file system that originates from on-premise storage. The virtual file system is very intelligent in that it doesn't deliver a "scatter" of all data to all endpoints. Instead, it retrieves as needed and is authorized for many situations. And here is where the policy can kick in: Certain endpoints and users can have the full contents of the virtual file system pushed out to avoid interactive retrieval.
In terms of the origination, the on-premise storage resource (accessed over NAS or object-based storage protocols) maintains the full content of the virtual file system and allows each authenticated user and authenticated device to access their permitted view of the virtual file system.
The key here is that there is a triple-factor encryption. The virtual file system is encrypted at rest on premise (presumably in your own datacenter) and the sub-instance that would be on an endpoint is encrypted as well. The authentication of a device is encrypted. Lastly, the user authentication process is encrypted both from the virtual appliance locally (again, in your own datacenter) to Active Directory.
There are a number of policy-based approaches as well to ensure that the instances of the virtual file system exist in a manner that makes sense for devices that are floating around the Internet and other places unknown. One of those is a cache timeout where, if the device and user don't authenticate back, the instance of the virtual file system will deny further access. Example here is if an employee is fired from an organization, but the PC never subsequently connects to the Internet. Then, the virtual file system can be set to prohibit the user from subsequent access.
In terms of the endpoints, Windows, Mac OS, iOS and Android are ready to go and for PCs and Mac systems the Oxygen Cloud appears as a local drive. That makes it easy for the user to access the data. Under the hood, it is a virtual file system, yet for mere mortals out in the world we need to make this an easy process.
What do you think of this approach? I like the blend of a smart virtual file system coupled with rich policy offerings. Share your comments on content virtualization here!
Posted by Rick Vanover on 09/12/2012 at 12:48 PM5 comments
Many of the vSphere 5 features seem to gravitate around storage, but there are scores of other features that can help you as you work daily in the the vSphere environment. One of those features, multi-NIC vMotion, allows multiple Ethernet interfaces for both standard vSwitches and vNetwork Distributed Switches (vDS) to perform vMotion traffic on multiple interfaces.
By default on a standard vSwitch with two network interfaces and one vmkernel interface, that traffic will only use one port on the vSwitch. With vSphere 5, we can add a second vmkernel interface to that vSwitch and configure adapter preferences to ensure that multiple NICs are leveraged. The first step to making this happen is to have two or more interfaces assigned to one vSwitch. From there, we can add the first and second vmkernel interfaces to that same vSwitch. The IP configuration has to be unique for each vmkernel interface, even though it will reside on the same vSwitch.
Once those steps are configured, we can set a switch failover over that has each vmkernel interface having one interface set as active; with the other interface set as standby. Reverse the configuration for the other vmkernel interface and there we have Multi-NIC vMotion. Fig. 1 show how this looks in vSphere for both interfaces, starting with the properties of the vSwitch (simplified in this example to only have vmkernel).
 |
Figure 1. Note that each vmkernel interface on the same vSwitch has different standby and active adapters. (Click image to view larger version.) |
Inside the configuration of the vSwitch we can look at the failover for each vmkernel interface. There we can see how this is configured by putting an explicit failover configuration for each interface. These failover configurations are opposite of each other (see Fig. 2).
 |
Figure 2. Each failover configuration points to opposite vmnic interfaces for each specific vmkernel interface. (Click image to view larger version.) |
If you are interested in seeing the step-by-step process to create this feature, check out VMware KB TV for a recent post on configuring this for vSphere 5 environments as well as KB article 2007467.
From a practical standpoint, this is an opportunity to get very granular with your separation to improve both separation and performance. It may make sense to leverage separate switching infrastructures, VLANs or IP address spaces to put in the best mix of separation and performance. It's tough to make a blanket recommendation on the "right" thing to do, but it is clear that this can help in both situations. From an availability standpoint, the vmkernel will be fine as there are multiple vmkernel interfaces; and the failover order (albeit manual) is still defined.
Does multi-NIC vMotion feel like a necessity for your environment? If so, how would you use and configure this feature? Share your comments here.
Posted by Rick Vanover on 12/08/2011 at 12:48 PM2 comments
VMware vSphere 5 ushers in a number of critical features which are very dependent on storage. So much so, that it has been dubbed the "storage release." One of the critical things to make the storage features come alive is VMFS-5.
VMFS-3 volumes are very much in play in today's vSphere environments. So much so, that the upgrade process may seem quite daunting to some. While we can simply right-click on a VMFS-3 datastore and update it to VMFS-5, we may be doing ourselves a disservice and missing out on this opportunity to perform some important housekeeping on our most critical resource: vSphere storage. The easy upgrade is shown in Fig. 1.
 |
Figure 1. A datastore can be upgraded, even with running VMs in place within the vSphere Client. (Click image to view larger version.) |
The migration to VMFS-5 offers a wonderful opportunity to knock out any configuration issues and inconsistencies with our storage that have been passed along thus far in our vSphere experience. So, I've collected some tips (and why they are important) to help make our transition to vSphere 5 a bit easier. I'm further making two assumptions that can make these tips easy to implement. The first of which is that we have some amount of transient vSphere storage. Secondly, we'll need some sort of storage migration technique. This is most commonly done through Storage vMotion or scheduled downtime and the migration task within the vSphere Client.
Now that those ground rules are out to set the assumptions, the next step is to identify the steps to make the upgrade to VMFS-5 volumes the cleanest.
The single biggest recommendation is to re-format each LUN to VMFS-5 rather than upgrade it. This will fix a number of issues, including:
- Mismatched block size: VMFS-5 introduces the 1 MB unified block size. We would be wise to avoid VMFS-5 LUNs with 2, 4 or 8 MB block sizes looking forward.
- Adjust geometry: We now won't be constrained to previous size provisioning. Further, we can resize LUNs on the storage processor to usher in new features such as storage pools.
- Correct zoning: This also is the right time to ensure that all WWPN or IQN access that's in place is still correct. Chances are that for old LUNs there may have been hosts removed from the cluster, yet they may still be zoned in the storage processor or switch fabric to the LUNs.
Reformatting each LUN to VMFS-5 with the correct zoning, a consistent 1 MB block size and "the right size decision right now" for each LUN will set the tone for a clean infrastructure from the storage perspective that can go along with the upgrade. This of course means that VMs will have to be evacuated from existing VMFS-3 volumes via a technology such as Storage vMotion to a (possibly) temporary LUN.
Outside of the storage provisioning metrics above, this is also a great way to clean up miscellaneous unused data on the VMFS-3 datastore. This can be VMs out of inventory, a forgotten ISO library or use of the VMFS filesystem for something other than holding a virtual machine.
It will definitely take some work to clean up VMFS-3 volumes in preparation for the best experience with the vSphere 5 features (such as Storage DRS). But, I believe it is worth it for any sizable SAN environment.
Do you see the upgrades worth the effort? Share your comments here.
Posted by Rick Vanover on 12/01/2011 at 12:48 PM1 comments
Upgrading from ESXi 4 to ESXi 5 can be challenging, especially if you're using the free version of the hypervisor. That's because for the free hypervisor, we don't have the access to the more robust vSphere Update Manager (which makes upgrading very easy).
There are a few options to upgrade a free ESXi 4 hypervisor to ESXi 5. These options are spelled out in the vSphere Upgrade Guide. In this post, I'll show one method and I'll mention the other mechanisms. Basically, the options include using vSphere Update Manager (which is available only with vSphere Essentials and higher), performing a scripted upgrade, using vSphere Auto Deploy, using esxicli or using the ESXi 5 CD.
For those who use the free hypervisor, by far the easiest option is the CD one. This is not very scalable, but it is very simple.
What happens is that the install media for ESXi is used to perform an upgrade of the installation and preserve VMFS volumes. While I like the option to preserve the datastores during an installation, I still think it would be a good idea to remove access to any volumes during an upgrade. In the case of a fibre channel SAN, simply disconnect the HBAs. If the storage is iSCSI or NFS, maybe remove the network path during the installation.
Direct Attached Storage can be a bit trickier, especially if the VMFS volume is on the same disk as the ESXi partition scheme.
It goes without saying (but I'm gonna say it anyways) that a good backup of the virtual machines is a good idea, especially if those VMs are on local storage!
Now, the first step is to download ESXi5 and write it to a CD-ROM disk. When booting from the media, the ESXi installation screen will appear. You may have to change the boot options on the system to start with the optical drive, since the local disk may already have a partition on it. The installation wizard for ESXi starts as shown in Fig. 1.
 |
Figure 1. The ESXi 5 installation wizard will start from the boot media. (Click image to view larger version.) |
The installation proceeds much like a normal installation, except you'll encounter questions about an existing ESXi installation and VMFS volume. You have the option to migrate the ESXi installation to version 5 and preserve the VMFS datastore (the default), which I recommend.
If you have fully evacuated the host, a clean install just becomes a better idea at that point anyways and the datastore will be rebuilt as VMFS-5. The critical options for upgrading the ESXi host are shown in Fig. 2.
 |
Figure 2. Upgrade-specific options are presented during the upgrade process. (Click image to view larger version.) |
It is important to note that rolling back via this method is not supported. This warning is presented during the upgrade process, so make sure that upgrading is the desired action. The upgrade will then proceed on the ESXi 4 host, and make it an ESXi 5 host. You may want to download the vSphere 5 client ahead of time for easy connectivity once the upgrade is completed.
With ESXi 4.1, we lost the ability to do the upgrades via the Windows Host Update Utility, but this process is pretty straightforward for the major updates from ESXi 4 to ESXi 5.
For the free ESXi hypervisor, what tricks have you employed in upgrading? Share your comments here.
Posted by Rick Vanover on 11/07/2011 at 12:48 PM1 comments