In-Depth
Do You Really Need Storage Management Software?
Local storage, virtualized storage, storage in the cloud. It's a mystery as to how we know what we have available on our sprawling systems. That's where storage management solutions come to the rescue.
There's been something of a debate as to the importance of storage management software. Some claim that it's by far the most important piece of an organization's storage infrastructure. Others claim that storage management software really isn't all that important, and you can get by without it. Given that this topic has been the subject of such spirited debate, I wanted to chime in on its importance and as I do, I will talk about some key features to look for, and I will talk about some of the features that you probably don't need.
Storage Management, At Its Essence
For those who might not be completely familiar with the concept of storage management software, it's a general term for a management application that helps manage and monitor all of the storage an organization is using. There are a number of different storage management vendors, and each vendor, obviously, has its own feature set. However, there are two basic capabilities that are more or less universal.
The first of these capabilities is unified storage management. Manually managing a few storage arrays is typically no big deal. However, keeping track of numerous arrays is a different story. Storage management software generally allows you to view all of your storage arrays through a single interface.
The second capability most vendors offer is storage reporting. The reporting engines vary widely from one vendor to the next, but, typically, the reporting software will show you the amount of free space on each array, which arrays are receiving the highest level of read and write IOPS, and which -- if any -- arrays are experiencing health problems.
Keep in mind these are the most basic storage management software capabilities. Many vendors offer feature sets that go well beyond the basics.
Hardware Considerations
The single greatest disadvantage to storage management software is that when you invest in it you might be paying for capabilities you already have. Some vendors offer storage management products with feature sets that are very similar to those that are built into most modern storage hardware appliances. For example, some products offer storage deduplication as a feature, but most modern storage arrays offer hardware-level deduplication.
Obviously, when you're speaking in generalities as I am in this article, there are few absolute truths. In most cases, however, if a feature is supported both at the hardware level and at the management software level, you're going to be better off using the feature at the hardware level.
Take storage deduplication, for example. If a storage appliance is capable of performing native storage deduplication, then the appliance probably has a CPU that's dedicated to the task of data deduplication. Because deduplication is being performed at the hardware level, you don't have to worry about burdening your servers by making them perform deduplication.
Another argument to using hardware-level features whenever possible is that doing so makes it easier to get technical support. Imagine the vendor finger-pointing that could occur in a support situation if your storage appliance natively supports deduplication, but you chose to allow the storage management software to handle the deduplication process instead.
While these are compelling reasons for performing deduplication at the hardware level whenever possible, there are a couple of important things to remember. First, I'm only using deduplication as an example. There are many other features that sometimes exist at the hardware level and as a storage management software feature. Second, there are no absolutes. Sometimes there are situations that make it more prudent to use a feature at the software level rather than enabling the feature at the hardware level.
To show you what I mean, let's go back to my earlier example of deduplication being performed either at the hardware or at the software level. Hardware-based deduplication has the advantage of offloading the deduplication workload from production servers (among other things). The disadvantage, however, is that the deduplication process doesn't occur until the data reaches the storage appliance (or possibly even after the data has been written if post process deduplication is being used). That's all well and good, unless network bandwidth is a big limiting factor. If your servers and your storage are separated by a low bandwidth link, then you're probably going to be better off performing server-side deduplication so that you can minimize the amount of data flowing across the network connection.
It's worth noting that some vendors also support a global deduplication plan in which data is deduplicated at the server side, and then a second round of deduplication occurs on the storage appliance as a way of eliminating cross-server redundancy (such as eliminating system files that exist on multiple servers).
Keying in on Features
So now that I've spent a bit of time debating whether redundant features should be enabled at the hardware or the software level, I'll go back and talk about some key storage management software capabilities. In doing so, I want to talk about some of the more beneficial features that you should look for in a storage management application, and why those features are important.
Unified Management Console
As previously mentioned, one of the key features of almost any storage management application is a unified management console that allows the administrator to see all of the storage hardware through a single pane of glass. Why is this feature important?
On small networks, it might not be. Such a feature might be convenient, but not essential. After all, centralized management delivers the greatest benefit when there are too many storage devices present for them to be managed manually.
So what about large networks? Some organizations might already have centralized storage management without having to invest in a third-party utility because some storage vendors provide storage management tools that can manage multiple storage devices.
Large organizations are likely to see the biggest benefit from a unified storage management console when they use heterogeneous storage. The management utilities provided by the storage vendors are nice, but more often than not they're vendor-specific. If you use storage devices of varying makes and models, a third-party, unified storage management console will probably be beneficial to you.
Reporting and Alerting
As previously mentioned, the reporting mechanism is another storage management feature worth considering. The utilities provided by storage vendors typically offer reporting functions even alerting capabilities. For a third-party storage management utility to be worth the price, it must have reporting and alerting capabilities that go beyond those provided by the storage vendors.
When it comes to a storage management utility's reporting capabilities, it's extremely important to make sure the reports that can be generated are actually useful to you. Over the years I've seen software vendors design products capable of generating an insane number of reports. Oftentimes I've found that when a vendor boasts about the sheer number of reports its product can produce, many of the reports aren't anything useful in real-world situations. Such reports are seemingly included only to make the product's reporting capabilities sound impressive.
In some cases, having a product that includes an excessive number of report templates can be counter-productive. I've used products that included so many report templates it became difficult to find the report template I wanted to use.
What types of reports should a good storage management product be able to deliver? Obviously, the product should be able to tell you how much space is free and how much space has been consumed on each array. You should ideally be able to view these figures in gigabytes or as a percentage (37 percent capacity remaining), and you should be able to retrieve this type of information for every storage device in your organization.
In order to be useful, the reporting engine should also be able to track storage consumption on a per-array basis, and should be able to warn you of potential shortages well in advance. Similarly, the software should help you track down any free LUNs within your organization so you can reclaim that capacity.
Although a storage management application should be able to assist an administrator with long-term growth and capacity planning, it should also be able to assist with ensuring reliability and performance. For instance, the reporting engine should be able to identify the busiest arrays or the busiest LUNs and help track down excessively high levels of resource contention in virtual server environments.
Ideally, a good storage management application should also be able to monitor your storage connectivity. By doing so, the software can tell you which links are being over-saturated, which ones are being under-utilized and how you might be able to move things around to achieve better overall performance.
As mentioned earlier, any storage management tool should be able to generate alerts to tell you when a LUN is getting low on space. However, "low on space" is a relative term. Some administrators might want to know when a LUN is down to 30 percent of its original capacity, while others might not want to be notified until only 10 percent of the space remains. As such, notification thresholds need to be customizable.
In addition, a good storage management utility shouldn't lock an administrator into a preset collection of rules and alerts. The software should give administrators the tools to build their own rules to go along with or to replace the built-in rules. Furthermore, the rule templates need to provide granularity. Granular rules might, for example, allow an administrator to more closely monitor storage that's used by a mission-critical application.
Finally, a good reporting engine should not only be able to tell you how much space you're using, but also how that space is being consumed. An administrator should never have to look at a disk and wonder where the space went. The storage management product should be able to tell them which users are consuming the most disk space, which data is consuming the most space, and what data was most recently added. Such reporting is relatively common for file server data, but some storage monitoring products are application-aware and can provide usage data for application databases.
Provisioning and Performance
While some storage management products act primarily as monitoring and reporting tools, others actually include storage-provisioning capabilities. There are storage management tools that will allow you to build a LUN on raw storage space, or even create a virtual hard disk.
Some of these tools take things a step further by offering features and capabilities designed to improve storage performance and reliability. I talked about deduplication features earlier, but deduplication is only one such feature.
Another example of a way some storage management products are able to improve performance is by taking control of storage routing. Some products are able to detect the connectivity paths that exist between a server and its storage and intelligently route storage traffic along the most efficient path based on the current utilization of each segment. Similarly, such products may offer multi-path I/O capabilities, which allow a server to take advantage of redundant links in order to improve overall data throughput.
Although such features are no doubt useful, they must be implemented with caution. I once saw an improperly configured system on which the hardware and the software both thought they were controlling storage traffic. The result was glacially slow performance and a lot of dropped packets. This was one of those situations in which an administrator needed to make a conscious decision as to whether routing should be handled by the hardware or by the software.
Some storage management products are able to create storage tiers that dynamically move storage blocks based on the current business need, in an effort to deliver the best possible performance. Obviously, each vendor does things a little bit differently, but the basic idea is that the storage management software keeps track of how frequently each storage block is read. If the system determines a block is in high demand, the block will be automatically and transparently moved to solid-state storage so the block can be read at the highest speed possible.
Automatic storage tiering is another feature an organization might be able to take advantage of without purchasing storage management software. Windows Server 2012 R2 has a built-in storage tiering feature that works in a manner that's very similar to what I just described. Similarly, some of the storage array vendors offer this type of functionality at the hardware level, either through solid-state storage or memory caching.
Although some of the features I've discussed might exist in your server OS or in your storage hardware, it's possible that storage monitoring software might do a better job of combining some features than what you might get if you were to use a piecemeal solution. Let me give you an example.
Earlier I discussed the deduplication feature. Although there are numerous variations, deduplication is typically based on the removal of redundant storage blocks. As such, some files will end up sharing a common copy of a storage block.
Now, imagine you also decided to enable a storage tiering feature that automatically moves frequently accessed blocks to solid-state storage. If one particular block is shared by multiple files, then that block might be used more often than some of the other blocks. By moving these shared blocks to solid-state storage, the system could potentially improve the access time for several files at once. This is what I mean when I say some product features might work especially well when used in conjunction with one another.
What Should You Do?
I'd like to wrap things up by going back to my original question: Should you consider investing in third-party storage management software, or is it OK to continue using your existing solution?
Small organizations can probably get by just fine without a storage management product. Similarly, large organizations that use storage hardware provided by a common vendor might be better off using the vendor's own storage management tools rather than relying on third-party tools. This is because the hardware vendor tools are going to be optimized for use with its own hardware, and might even expose features that a third-party management solution would not. But for midsize-to-large organizations with storage products from multiple vendors, a good storage management product would probably make the administrator's life much easier.