In-Depth
Storage Wars
Storage is easier to use, but still not affordable for SMBs.
In the past 10 years the storage industry has seen an explosion of innovation. Far from hyperbolic cheerleading, the effect is quantifiable. Innumerable startups have entered the space, many with radical new approaches to storage, and they have transformed the entire market.
The big winners of the storage wars have been object storage, hyper-convergence and scale out storage. Flash, as a storage media, has clearly benefitted as well, but flash has merely been an enabler; it hasn't provoked a fundamental shift in how we store or access data.
Somehow over the past 5 years, through no intention of my own, I have been drawn in to the storage industry. I have been accused of being, at varying times, an analyst and various flavours of expert. It's true that I have tracked the storage industry closely, but this was never purposeful. My interest in storage is a defence mechanism. I am still an IT practitioner: and, to put it bluntly, storage sucks.
The Why of the Storage Wars"Storage sucks" is the core sentiment behind why the storage wars happened in the first place. To say something sucks is pretty generic, so let's look at the two primary measures in which the storage of 10 years ago failed, and see if all that innovation really helped at all.
The first thing storage was awful at was convenience. There's an obligatory XKCD on the topic, of course, and it's a lot closer to home than most in the industry would like to admit. Whether the task is sharing files between two people, or assigning shared storage to a virtualized workload, the storage of 10 years ago was a monumental pain in the proverbial.
Although to some extent this has changed, the XKCD comic is still true: sharing files between two people remains a pain. We have, however, made advances in ease of use when it comes to administering shared storage. Most hyper-converged solutions are great examples of how storage can be made easy to use. Similarly, virtual machine (VM) and container-aware storage solutions that integrate with virtualization management interfaces can remove the headache of dealing with storage for everyday tasks.
We haven't made storage easy to use for everyone, but 10 years of innovation have made it easy to use for most systems administrators. As a systems administrator, I'm going to give that a great big thumbs up.
One of the other means of interpreting the phrase "storage sucks," however, doesn't seem to have improved. Here I am talking about reliability.
Storage Still Sucks
I don't care what anyone says, they'll not convince me that 10 million files and 20-some terabytes of data is "a lot" these days. Every single SMB I work with generates at least this much data. Most are in the 100TB range, with 50 or 60 million files.
The problem here is the term "SMB:" small and medium businesses. It's the overwhelming majority of the market. These are regular, everyday businesses that aren't swimming in cash, and can't afford to pay $100,000 for a 20TB NAS (Network-Attached Storage). Especially when a $10,000 NAS claims to offer 40TB of storage and do everything the $100,000 NAS can do.
The issue is that the $10,000 NAS doesn't do so well with 10 million files. It doesn't do well with 2TB files (like virtual hard disks). It certainly doesn't like compressing and encrypting them and then firing a copy up to the cloud. These perfectly mundane operations "stress" the low end NASes and cause them to freeze, crash, randomly reboot and otherwise behave unreliably.
This isn't to say the $100,000 NASes do much better; they don't. But for the extra $90,000 you'll get someone on the phone who pretends to care, and a series of repeated visits by someone who will tell you "it looks fine to them". Eventually, six months down the line, maybe someone at the vendor will identify a bug and fix it, only to introduce two more in the process, or tell you to use their new object storage solution instead.
Picking Winners
Basic file storage is one example of how the storage wars haven't helped the customer. Basic problems haven't been solved. Or they have been, but operating system vendors haven't "picked a winner" by embracing next generation solutions and integrating support into the OS.
Until the OS vendors pick those winners, the storage wars look set to continue.
About the Author
Trevor Pott is a full-time nerd from Edmonton, Alberta, Canada. He splits his time between systems administration, technology writing, and consulting. As a consultant he helps Silicon Valley startups better understand systems administrators and how to sell to them.