News

An Alternative to Microsoft AppFabric

ScaleOut says its software can do more with its "in-memory data grids."

• By Kurt Mackie
• 11/17/2015

ScaleOut Software today announced an enhancement to its ScaleOut StateServer product for organizations looking to replace Microsoft's AppFabric Web caching technology.

Microsoft had announced back in April that it planned to deprecate AppFabric 1.1 for Windows Server, but it gave the product another seven years of life after getting customer feedback that they needed more time to move. In its place, Microsoft has recommended that organizations use the open source Redis for Windows Server solution or its own Azure Redis Cache service.

It turns out that organizations have other alternatives as well. Bellevue, Wash.-based ScaleOut Software, a Microsoft partner, has its view.

Today, ScaleOut Software announced that it is introducing a new AppFabric Compatibility Library into its latest ScaleOut StateServer product that will ease the transition away from AppFabric. In addition to making the switch easier, the ScaleOut StateServer product offers more advanced capabilities compared with Redis and AppFabric, according to company officials. For instance, ScaleOut's solution supports Microsoft LINQ querying. It has extended APIs. An "object browser" can be used by developers to debug data in a grid. The solution supports C# MapReduce. And it works with public cloud services from Amazon or Microsoft.

I spoke earlier this week with Dr. William L. Bain, CEO and founder of ScaleOut Software, as well as Chris Villinger, vice president of business development and marketing for the company, who are both Microsoft veterans. The company currently has more than 420 customers and its products, supporting both Windows and Linux, are running on more than 10,000 servers. The company's ScaleOut StateServer product was first introduced into the market back in 2005.

Bain explained that he had started ScaleOut Software in 2003 after leaving Microsoft. He had developed load balancing software that Microsoft acquired after it bought his earlier company, Valence Research, and that network load balancing technology has been used in Microsoft's products from Windows Server 2000 and thereafter. Microsoft's AppFabric was first introduced into the market in 2008 under the "Microsoft Velocity" name, adding other features besides caching. It was similar to the technology developed by ScaleOut's founders.

Q: What is ScaleOut's technology?
Bain: We developed this technology that at the time we called "distributed caching." We've since renamed it as "in-memory data grids." It's a software middleware technology that sits above the operating system and beneath the application. And its goal is to store fast-changing states in memory across a farm of servers that are clustered, often collocated with Web servers or applications servers, but also can be resident on a separate caching farm. And the purpose of this is to allow scalable access to fast-changing data and to integrate high availability into the in-memory data storage, as well as to provide for in-memory computing data parallel computation, like MapReduce, on the data that's changing in memory. And the reason why this technology is important is because without an in-memory, scalable, highly available data store, then there has to be another alternative place to store fast-changing data of the type I just mentioned, and that usually falls to a database server like SQL Server. And SQL Server or another centralized data store can quickly become a bottleneck as a server farm grows, as the amount of data being stored and the amount of data that's changing rapidly increases.

Is Redis for Windows or Azure Redis an adequate replacement for AppFabric, as recommended by Microsoft?
Villinger: We were very confused by the direction Microsoft is taking. It doesn't make sense in many cases to push customers to their cloud-first philosophy by going to Azure. We feel that we responded to the request of the community by offering a very simple path, with additional powerful features and capabilities to move from AppFabric to a more full-fledged distributed cache. Redis confuses us because it's like apples and oranges. Redis has a very specific place and it does what it does extremely well. But then to extrapolate it to a full-fledged, scalable, highly available data grid, that's not what it's designed for and it can't do those things. So we don't understand the recommendation.

Bain: Redis was not conceived as a scalable, highly available, in-memory data store in software. It was instead conceived as single server in-memory store for data structures. So its real claim to fame is its ability to store specialized data structures such as sorted lists, hash tables and other specialized data structures that could be updated with low latency on a single server. It was not designed to scale and provide high availability. Now the open source implementation of Redis that Microsoft has put on GitHub and made available to its users that are coming off AppFabric has those same properties. However, version 3 of Redis is now just beginning to introduce clustering. And they have a high availability story with Redis in version 2.8, and adding scalability with clustering in 3.0. But this is a brand new feature set for them and it lacks many of the key features that make ScaleOut StateServer a premier in-memory data grid providing transparent, easy to use, highly powerful data storage in memory and in-memory computation.

What does ScaleOut StateServer offer above Redis or AppFabric?
Bain: AppFabric caching does not provide some of the more advanced capabilities that we have developed and integrated into our product line over the last decade. We have full property-based querying using Microsoft LINQ (the query mechanism in C#). That's simply not available in AppFabric caching and it's certainly not available in Redis. We also have other features in our architecture, being a fully peer-to-peer architecture, that's designed to have transparent scalability. You simply add a server and the workload is migrated to that server to keep a balanced workload, no matter how many servers you have in the cluster or server farm. Also, as you have failures, it automatically detects and handles those failures. This sophisticated transparency that we've built in for ease of use with our technology is simply not available in Redis. Redis was really not designed from the ground up for scalability and high availability, and while AppFabric does tackle those two requirements, it provides a fairly basic feature set that we have made available with our AppFabric caching Compatibility Library, but then our technology goes well beyond that basic set of features to deliver a really powerful in-memory data grid with in-memory computation as well.

What does it take to replace AppFabric with ScaleOut StateServer?
Villinger: Imagine a case where an application has been running in an AppFabric environment for some time. The development team for that application has been disbanded and IT is now in control. They don't want to completely rewrite the application. In that type of scenario, you can come to us and use this drop-in Compatibility Library to just point your application away from Microsoft AppFabric and to our in-memory data grid. You have to make one little change to make the application look at the correct grid and then it configures itself and it has automatic management. So organizations can use their existing code and continue to run the application as is. Now, in most situations, though, there will be some development or support development teams available, and you'll probably want to take the opportunity to make improvements to the application, in which case you can use our data APIs as well in parallel. From an IT perspective, it uses all of the same familiar PowerShell commands to manage the grid, all of the same API calls that you're using in AppFabric. So it's a very seamless transition.

Bain: ScaleOut's solution is fully source-code compatible with an AppFabric application, so any Microsoft C# developer can simply take our Compatibility Library and connect it into their Visual Studio environment and then recompile their application to use the AppFabric caching Compatibility Library. And the APIs for AppFabric are made available by ScaleOut. In many cases, we improved upon the implementation provided by AppFabric itself. For example, the AppFabric eventing model is not highly available. We made it highly available and fully scalable in our implementation because of our architecture. But the second big benefit to moving to ScaleOut StateServer to use the AppFabric caching library is that it's much easier to deploy and manage the cluster, the distributed cache, with our technology. AppFabric caching had some inherent complexities built into it. In particular, it had to be woven in fairly tightly into the domain environment and into the permissions environment in Microsoft server clusters in order to make it work. Whereas, with our technology, it's designed to be installed and automatically self-configuring and automatically scaling, automatically self-healing in case of a server failure. We eliminated, for example, the need to use a centralized configuration store on SQL Server or a file share, which AppFabric caching requires.

What are the tools used with ScaleOut StateServer?
Bain: We have our own scripting tool that provides full capability for managing the cluster and the distributed cache. In addition, we have a centralized GUI console and you can do things with that that are much more difficult to do with a command-line PowerShell cmdlet. For instance, there's performance charting to show how the grid is doing and each server. There's a heat map to show which parts of the data are more heavily used. One of the problems you have with distributed software, software that runs across multiple servers, is having a holistic view of the entire cluster. And a centralized GUI console lets you see at a glance the health of the cluster. We have an object browser. Developers and administrators can use it and drill down to look at the data into the grid. They can pick apart objects and see their properties and values. You can look at a shopping cart and see all of the items put into it when it's live.