In-Depth
I Got Those Bulk Storage Blues
Serving the extremes at the expense of the masses.
I like big storage, and I cannot lie. I want terabytes and petabytes; exabytes piled high into the sky. I want bulk storage and it doesn't have to be fast; bulk storage I demand; my data has to last!
The key in my horrible attempt at IT poetry are the terms "big" and "bulk." I want to store a lot of data. I don't necessarily want to store that data rapidly, nor do I need to access it particularly fast. The problem is that no vendor wants to sell me a reliable solution for my needs at a price I can possibly afford.
History Lesson
For a long time, the IT industry has been built on making something that was faster than the previous something. CPUs sold this way until the laws of thermodynamics and the practicalities of lithography meant that single-thread performance hit a wall. Increased performance simply generated too much heat.
We built multi-core processors. Chip vendors tried to convince the world this was a good thing and dumped billions of dollars into trying to bring up a generation of developers who could actually make use of the new silicon. Fifteen years later we finally have enough of those developers available that we're seeing a boom in machine learning, neural networks and other parallel-processing-optimized fields.
Storage is in a similar place. Long gone are the days where performance and capacity of rotating media grew by leaps and bounds every couple of years. Disk speeds plateaued years ago and disk capacity growth is stagnating.
Flash is really only the solution to speed, and that solution isn't coming with big discounts for customers. There are absolutely workloads where speed matters more than capacity; the problem is that these aren't the majority of workloads, they're just the most attractive to vendors.
It's easy for a flash vendor to walk into an account using magnetic media and sell flash. One can take a 24-disk RAID 10 of 15K RPM rotating media and absolutely crush them with two NVMe SSDs in RAID 1. There is no question: flash is faster than hybrid which in turn is hilariously faster than disk. If the customer cares only about speed, you can sell them a two drive flash solution for 75 percent of the price of the 24-drive disk solution and pocket a mind-numbing amount of profit.
The Fuzzy Middle
Bulk storage comes in one of two forms: flash or spinning disk. Flash bulk storage, based on QLC flash chips, is a new phenomenon. Drives up to 16TB are available. They are, however, priced in the "if you have to ask, you can't afford it" range. For mere mortals, rotating disk is the only bulk storage available.
Because disk speed isn't growing with capacity, RAID rebuild times are horrific. RAID has moved beyond risky to become unusable for bulk storage in practice, leaving erasure-coded object storage pools to take up the slack.
Unfortunately, few applications and operating systems speak to object storage natively; file is still the access solution preferred by nearly everything, especially hypervisors. Storage software that can do erasure coding and deliver a file interface stable enough for virtualization is rare and expensive, making "deep and cheap" storage in many ways harder to achieve today than it was five years ago.
Cloud storage isn't exactly cheap, either, even if you ignore the elephant in the room of the cost of fast, reliable Internet connectivity.
If you need a few terabytes of storage, a small two-disk RAID 1 will probably meet your needs. If you need half a petabyte, an object storage solution will offer a usable cost per gigabyte that most would find acceptable. The fuzzy middle, however, is where the majority of the world's businesses live.
Ignoring the Mass Market
The cost per gigabyte of 10TB - 500TB of reliable storage is awful. No vendor wants to play here; that's the "volume play" market, where margins are slim and vendors have to work for a living. The same argument can be made about those who only need to light up between 10 - 500 virtual machines (VMs); the cost per VM of a resilient and redundant solution is pretty abysmal compared to the cost per VM for 1-10 VMs or anything above 500 VMs.
This is IT in a nutshell. The extremes are well served, but the mass market never is. It was always thus and does not look set to change.
About the Author
Trevor Pott is a full-time nerd from Edmonton, Alberta, Canada. He splits his time between systems administration, technology writing, and consulting. As a consultant he helps Silicon Valley startups better understand systems administrators and how to sell to them.