The Infrastruggle

The Twilight Zone of Storage

The numbers being tossed around by storage analysts are not adding up in this weirdest of dimensions.

• By Jon William Toigo
• 05/14/2014

It's like one of those old Twilight Zone episodes. The main character awakens in an unfamiliar place, usually in a hotel room in a small town at night, only to discover that he is completely alone. He tries to reach the front desk with the bedside phone, but receives no answer. Seeking an explanation, he dresses and begins exploring.

He strolls through the vacant lobby, then pushes out through the front door and onto Elm Street. The storefronts are all there, but the diner and shops are devoid of customers and workers. The constant glow of street lights mixed with the dancing animation of the movie theatre marquee chaser lights reflect off the fenders of pristine, late model automobiles parked at the sides of the road. But, there are no drivers in the cars, no pedestrians on the sidewalk, no traffic at all.

In the distance, the sound of music playing on a jukebox emanates from some ice cream shop or local bar. Then, suddenly, the air is pierced by the bell in the courthouse clock tower as it tolls out the hour -- 9 p.m. 

Our character seems to be the only person around to hear the chimes. Pausing to consider the situation, he realizes that has many more questions than answers.

These days, reading storage industry analyst reports and projections engenders the same kind of eerie, disconnected feeling.  Nothing makes sense. Numbers don't add up. And sometimes you wonder if you are the only one around who even notices.

Just last summer, IDC altered its 2011 prediction regarding rates of growth in storage capacity demand -- going from 40 percent per year to 300 percent annualized growth in virtual server environments. We can assume that this projection reflected the emergence of vSAN and other server-side storage topologies, which blend direct-attached (or internal) storage with shared storage array architectures in a sort of clustered HA configuration. For such strategies to work, data must be replicated multiple times across these storage kits to ensure that virtual machines can move from physical server to physical server, finding at each perch a local copy of their data. VSAN strategy amounts to a storage capacity demand multiplier that explains IDC's 300-percent estimate.

(Technically speaking, only two data copies are required for vSANs to deliver HA, according to vendors, but experts argue that three data copies provide greater resiliency. So, 300 percent growth in capacity demand is what you can expect as a side effect of the movement toward hypervisor computing with server-side storage, argue the analysts at IDC.)

A few months later, Gartner upped the ante asserting that capacity demand would grow by 650 percent year over year, noting that: 1. We were also going to need to back up all that replicated data and would require some space to store the backups and, 2. We would likely want to replicate our data over distance to another data center (perhaps in a public cloud) to provide a way to re-instantiate VMs if the primary data center melts down. So, we would require still more capacity.

Okay. So, we were all set to see a spike in capacity demand driven by that trend toward virtualization of server workloads, which both IDC and Gartner claimed would grow from about 45 percent to nearly 80 percent of all server workloads from 2012 to 2016. But, this story of capacity demand doesn't appear to be supported by the facts of actual storage equipment sales.

In the first quarter of 2014, EMC storage revenue is 3 percent lower than last year, down by 22 percent quarter over quarter. IBM storage hardware revenues are also down by 23 percent Q/Q. Equipment sales throughout the storage industry seem to be off, typically by even greater percentages than we see with the big players.

Moreover, in the first quarter of 2014, sales of disk drives worldwide were down 4 percent compared to sales in the same quarter of the previous year (also a year of falling demand). By most accounts, the future doesn't look terribly bright for a resurgence in either disk or enterprise array sales. And what sales are occurring are often hugely discounted below MSRP.

As a consequence, the analysts at Wikibon are projecting the end of networked storage as we know it and the ascendency of "software-defined server-side storage." In other words, the SANs we spent ten years building would be disaggregated.

But that leads to another instance of Twilight Zone-ish conundrums:  The adoption of server-side should drive up acquisitions of hard disks and, perhaps, SSDs, to facilitate all of that "behind-every-server" data copying and data storage.  But, from the standpoint of equipment sales, we haven't seen this growth.

Some might argue that this is because of the "software-defined" aspect of the infrastructure, conflating "software-defined storage" with "virtualized storage." But, the two are not the same, according to SDS advocates.

While virtualized storage could provide planners a flexible alternative to breaking up the SAN, presenting virtual volumes to virtual machines that could move with workload from physical host to physical host without requiring back-end reconfigurations of routes to physical data, a recent survey commissioned by Virtual Instruments says that only 15.5 percent of the storage behind virtual servers has been virtualized.

Storage virtualization involves aggregating the physical storage capacity of multiple arrays so that combined capacity can be sliced and diced and served it up to application workload as "virtual volumes." That isn't the same as what software-defined storage does, according to SDS purists. Software-defined storage is simply a software controller layer that abstracts value-add storage services away from storage array hardware controllers.  Deduplication, replication, mirroring, etc. are all services that can be more effectively parsed out to application storage from this centralized service repository.

Simply put, SDS is not virtualized storage because there is no capacity aggregation tomfoolery involved in SDS. As a consequence, software-defined storage does nothing to shape capacity demand.  Thus, the software-defined nature of server-side storage offers no explanation for why the increased demand for storage capacity has not created an uptick in storage hardware sales.

Perhaps this is because the server virtualization that is supposed to be driving increased capacity demand isn't as prevalent as analysts think.  While projections of percentages of workload being virtualized annually sounds pretty impressive (virtualized workloads increasing at 45 percent per year), consider that the same analysts have pegged the number of physical servers that remained un-virtualized (e.g., that are not running hypervisor software) at nearly 90 percent in 2009, declining to about 83 percent in 2012.  Last year, they projected that the total percentage of physical servers not running hypervisors would fall to 79 percent by 2016. 

Putting the two estimates together, that means virtualized workloads will grow at 21 percent annually through 2016 while the number of physical servers that are running hypervisors will grow by a much more modest rate, 1.25 percent per year, in the same timeframe. By that calculation, 95 percent of workloads will be processed using only 22 percent of physical servers in 2016. The other 78 percent of servers will be processing single, non-virtualized workloads only.

Since the preponderance of servers will not be running any virtual machines, maybe that will mitigate the need to break up or segregate traditional SANs or to replace them with server-side storage configurations augmented by hundreds or thousands of data replication processes consuming a significant portion of the bandwidth of LAN interconnects.  In other words, maybe the SAN-pocalypse projected by Wikibon and others isn't happening in the next few years. Maybe the deconstruction of SANs and their replacement by server-side architectures won't be seen in the near term at all.

Where are the analysts looking at these contradictions and struggling to make sense of storage as it exists today? I am wondering about this as I wander through the vacant corridors of a virtual world. Am I really awake or is this small town just a dream?