News

Understanding Azure ExpressRoute

A Q&A with a Microsoft executive, who explains why some organizations would benefit from the service.

• By Kurt Mackie
• 09/18/2015

Public Internet connections aren't good enough for some organizations, contends Microsoft. For those situations, Azure offers a solution, known as ExpressRoute.

Someone who knows a lot about that is James Staten, Microsoft's chief strategist for cloud and enterprise. Staten joined the company recently from research and consulting firm Forrester Research, where he was a cloud computing analyst. He now helps organizations figure out how to bring together premises-based computing environments and cloud-based services in a "hybrid" strategy.

1105 Media Senior News Producer Kurt Mackie spoke with Staten at Gartner's Catalyst event in San Diego, where he added his expertise to a panel on "Interconnection Oriented Architecture." The panel was moderated by Redwood City, Calif.-based Equinix, a colocation service provider, as well as a partner with Microsoft on its Azure ExpressRoute connection service.

Mackie asked Staten to explain the circumstances for using Microsoft's ExpressRoute service, which enables high-bandwidth, low-latency connections that bypass the public Internet. What follows is an edited Q&A on the topic.

Q: Why would an organization need a private Internet connection using Azure ExpressRoute?
Staten: Typically, they'll do so for two reasons. The first reason is in the name itself, "private." They want their traffic not flowing over the Internet so therefore it can't be inspected, or tracked or copied by any unknown third parties. The second reason is for performance. So, through these connections, they might have traffic patterns that are unpredictable, and they want the ability to get a guarantee when they need it. So, for example, next week, during these two hours, I'm going to have to move a petabyte of data. I don't want that to take seven hours. I want to know exactly how much bandwidth I can buy from you and I want the quality of service that there's going to be this much throughput per millisecond.

So these connections are about moving petabytes of data. What sort of operations are people trying to support when using ExpressRoute?
There's unified communications, backup, batch and then there's Big Data analysis, so Internet of Things type of analysis as well. What often happens over an ExpressRoute connection is you will set up subchannels in that ExpressRoute connection for different traffic types and application types where you set separate quality-of-service guarantees on the different traffic types. So, most recently, we announced that ExpressRoute now works with Office 365. And so you could say, "I'm going to have my Exchange connection here, and then I'm going to use another sub-channel of the ExpressRoute connection for the other applications, but I want to make sure that those connections are really solid." Well, I'll take 20 percent of my ExpressRoute connection and I'll set a quality of service because I'm going to use Skype over that.

Because, if you're doing voice, or you're doing live video, your packets have to come in order. Packets have to be spaced appropriately. The last thing you can afford to do is have gaps in the conversation, and people complain that it's the software when it might have been the connection. And so that's a big reason that people will use an ExpressRoute connection.

Will most companies use these connections or just certain companies?
Most companies are going to see the value in the different types of connections that are out there and have a mixed approach. You're perfectly fine to do VPN where you don't care about quality of service and you know that you're going to secure the connection. In other cases, you really do want a hard line. Where I have a need for dedicated connection, a highly secure connection, a high-throughput connection, then I know I can go to a VPN up to a certain level and then I can switch over to an ExpressRoute from that point forward. We mostly see ExpressRoute connections between Equinix colocation facilities, where customers have a significant amount of footprint, and the closest Azure location, called regions.

How does an organization with a hybrid network use ExpressRoute?
The simplest way to think about it is, what are the two endpoints of my hybrid connection that I really want to concentrate on, first and foremost? That endpoint might be Equinix because that is my datacenter and I don't have another datacenter. In that case, I'm going to say, "OK, from Equinix Chicago to the closest Azure datacenter -- let's put the ExpressRoute in there." They can buy that from us, and then they now have a dedicated link between the two locations.

Now, part of the value that comes from that is, with the next five applications they deploy in a hybrid model, they won't have to set up five more ExpressRoute connections. They'll be able to simply add those applications to the existing ExpressRoute. They might have to increase the bandwidth of the ExpressRoute connection because there's more going on, but there's no other connection that has to be created. If you didn't have ExpressRoute and you just used the public Internet, between that Equinix facility and the Azure datacenter you would be setting up separate VPNs for every single application because they are going to go over the open Internet, which can be routed anywhere, so you don't have a guarantee and you don't have the protection.

The public Internet is less expensive and it's easier and faster to set up, but then I have to manage all of these VPNs, whereas if I can route everything over the ExpressRoute, I have one connection to monitor, one connection to log and one connection to secure -- and that's a much easier model for companies to do. So that's why we tend to see companies, once they have got a lot of hybrid activity, migrating to an ExpressRoute.

Do organizations face bandwidth issues trying to push a lot data across these networks?
It's not so much that the public Internet doesn't have the bandwidth. It's that you're usually dealing with noisy neighbors. Lots of other people who are using the same Internet connection as you are, so that's where you will tend to see some slowdowns.

These ExpressRoute connections are fiber-optic connections?
In most cases, it's what's called an MPLS (Multipacket Label Switching) line. And, in most cases, that is a fiber line between them. It's not universally the same everywhere, but that's mostly what you see in place.

Azure has service zones. The United States and Europe represent one zone, but, if you have to connect to Asia, then that's another service zone. Is ExpressRoute used for this Azure interzonal connectivity?
Between the Azure regions, we have the Microsoft private network. And so there's no separate ExpressRoute that's required. So think of the Azure regions as the sort of the center circle and then Equinix is sort of the second layer of circles. And the datacenters are the third layer of circles. ExpressRoute is between the layers of the circles rather than within that circle. So if you want to geographically redundantly lay down an application, where one's in China, one's in India, one's in Europe and one's in the United States, you do that all over the Microsoft private backbone via Azure. There are no unique connections that you have to set up between any of those things.

What can we expect down the line that Microsoft is doing with Windows Server technology to make it easier to connect with Azure?
Well, Azure Pack becomes Azure Stack, so it's an evolution of that product. And the biggest thing that's going to change in that architecture is that we are going to support containers like we do inside of the environments. If you want to use containers instead of virtual machines for the workloads you put in place, you'll have that flexibility and environment. And we're also going to add the Azure App Service to that stack as well so you'll be able to run the Azure App Service on premise. We will try to bring over as many of the other services we can that don't require hyper-scale datacenters to achieve.

Machine learning is something we've have a lot of customers ask about, but there are certain algorithms that run on thousands of servers at a time -- it's kind of hard to bring on premise. But we'll see what can bring over as part of that. Whereas Azure Pack is really kind of an infrastructure-as-a-service lightweight layer, with Azure Stack you're going to get as much of the full Azure stack as is possible to acquire on premise.

Now the other thing we're doing to help customers in the hybrid sense, is that they also are recognizing that if I want to put an app in containers, containers run on a shared OS instance underneath. That's been true since the early days when Sun and IBM created containers. And if that's the case and the container is going to access a few of the shared resources, but the underlying OS is really just going to be a layer to support containers, does it really need the full OS? And, if it doesn't need the full OS, can we do something that's a little more lightweight?

And that's what we're doing with Nano Server -- it's to provide a lighter weight framework that has the foundational stuff it's going to need in this role and you can optionally install the other things. And that will help address the density of applications per server. So we've already seen people talking about 10 to 15 virtual machines per server. Containers should take this up to about 30. If you go with Nano Servers and thin operating system models, you might be able to take that to 60 or 80 virtual machines or even higher. And so this is where we are going to help our customers really use their resources more efficiently.

And it will help automate server management with PowerShell?
Yeah, we are making sure that since PowerShell is our standard way for doing command-line interface and scripting management at the lowest level that this works on all of these versions of Windows Server. So you should be able to use the same PowerShell commands to a Nano Server as you would to a full Azure stack implementation, as well as to a VM you are running on Azure. The other thing that we are doing, as well, and you'll see this come out in the fall in Azure first, is that we are making modifications to Windows Server 2016 so that we can apply patches to the underlying operating system without taking the server down.

And those patching improvements are enabled through container technology?
Yep, that's a new capability that we're building directly into the operating system. Far less restarts are required. It's much more packet specific. The operating system is no longer this big monolith but it's a collection of services. And over the years we've been making the services independent of one another.