Big Data, Big Cloud, Big Deal

When I think of big data I don't really think of the cloud. First I think of big servers and big processors. Then I think of in-memory where this data is in RAM for lightning fast analysis. This sure ain't the cloud.

The other problem is big data doesn't fit well over the clogged pipes that lead to and from the cloud. However if your big data is somehow already all up in the cloud and if you are just sending instructions to manipulate it, then you don't have to buy the big servers with the big processors. There, the cloud might just make sense.

Rand Wacker from CloudPassage knows more about this than do I and took a whack at these issues in a recent interview with Enterprise Systems Journal.

Besides putting your entire big data app in the cloud, and there are already big data SaaS apps that can do the job, Wacker has an interesting hybrid alternative. "Instead of provisioning your computing infrastructure for peak load -- and then some -- you can cover the base load with your own private cloud and either lease the spikes or burst out into the public cloud when additional temporary resources are needed, as it is often the case with big data projects," he explains.

When it comes to having your all big data all in the cloud, Wacker is also a fan. "The on-demand nature of the cloud enables companies to very easily secure the amount of processing power they need, no matter how large or how small. Instead of building a data center dedicated to data analysis, companies can lease servers by the hour. For highly variable workloads, metered (or utility) billing matches costs to usage and can lower overall investment, especially for firms that are just beginning to leverage big data technology."

Does big data belong in the cloud or will we just overload the thing and slow down ESPN.com?

You tell me at [email protected].

Posted by Doug Barney on 09/18/2012 at 12:47 PM