Tracking enterprise infrastructure's evolution in the AI era

Building a Cheap, Screaming Virtualization Lab Server

As most of us know, today's home labs, and even a number of corporate labs, are pulled together with whatever parts users can get their hands on and/or recycled hand-me-downs. Typically this is the easiest, fastest and cheapest way to stand up a server lab for testing and engineer use. However, what's gained by getting things set up quickly is lost in power and effectiveness. How many of us in IT wish we could get an affordable system that also does what we need it to do?

Posted by Jason Mattox on 03/14/201214 comments

Tips for Migrating to Exchange in the Cloud

With all of this "cloud speak" these days, I figured I would give some real-world advice about what is involved in moving to a hosted Exchange solution in the Cloud. With our recent growth and staff additions here at Liquidware Labs, we made the decision to upgrade our mail server from our locally hosted Exchange 2003 server to a more robust and flexible cloud-hosted solution. We chose Microsoft Online services for our Exchange Cloud for a number of reasons, with the primary reason being cost. At $5 per user per month, with 25 GB of storage per user, the savings would be hard to beat. This service from Microsoft is now known as Office 365, and we will be completely migrated later this year to the full suite.

Posted by Jason Mattox on 12/09/20110 comments

Virtual Desktop Backup Options for VMware View

Why is it so hard?

Right now there is no seamless back-up method for VMware View. Nonetheless, the product has some real value. It is an asset for disaster recovery programs because it provides portability so desktops can be moved to the datacenter. How easily these virtual desktops can be backed up depends on how the virtual desktop infrastructure was created in VMware View. I'll explain the options and back-up approaches.

With VMware View you have two main methods for creating VDI sessions. One is to create the run-of-the-mill VMs that you're already used to. The second method is to create a linked clone, which is a snapshot of a virtual machine that shares virtual disks with the parent VM. Each method has its advantages and limitations.

VDI sessions from common VMs usually result in storage requirements and costs that are tough to swallow for virtual desktop environments of much size. However, this type of virtual desktop is easier to back up than a linked clone. If you have normal VM images that are static per user, you should be able to use your image-based backup tools that were designed for VMware from the ground up to back up these images. For restores, you might have do some work in VMware View to get the virtual machine back into the View inventory so the broker can hand out the VDI session again, but that shouldn't be a big deal.

Things start to get tricky when you look at linked clones. First, you need to understand that a linked clone is a snapshot with its own virtual hardware – which is different than a normal VM. With a virtual desktop population, you end up with a base disk that holds many, many snapshots, and each snapshot is a unique VDI session with its own IP address, name and numerous other details.

If you choose to use them, you can set up link clones one of two ways:

1. As a static linked clone that saves the user settings and data on logout, or
2. All changes are discarded on logout. If all changes are discarded on logout there isn't much reason to back up the linked clone. You'd only need to back up the base image, which you can do via the datastore browser by hand.

When you go to back up a linked clone, there are a few issues that come up with VMware and its API. To start, since the ESX host and Virtual Center are unaware of the linked clones, you can't use the vStorage API to back up these VMs. The second problem is that only VMware View knows the linked clone mapping, so trying to trace this back would require you to use an API from VMware View (if one exists), or dig into the VMware View database manually. Neither of these are good options from a back-up point of view. VMware needs integrate the linked clone data into Virtual Center and expand the vStorage API itself.

Those developments could come in the future, but what about backing up virtual desktops now? When using the vStorage API today to back up, it does a good job of putting an abstraction layer on top of the VM snapshot. This allows a back-up solution to grab blocks as if the snapshot didn't exist. However, on restore you lose the snapshot itself and end up with a monolithic disk, that from a data point of view is the most up to date, but has no snapshot on disk for you to re-attach. This works fine for VDI sessions that are NOT linked clones. However, VMware needs to expand the vStorage API to allow the snapshots themselves to be transferred, or allow the snapshot data be transferred back into an existing snapshot file, but vStorage API does not support these capabilities today.

So, for now if you have normal flat VMs with a static mapping you can back those up normally. But if you're using linked clones, the only real way to back up would be with hardware snapshots on LUNs that hold a small amount of VDI sessions.

Posted by Jason Mattox on 03/08/20115 comments

Cloud Storage Gateway Benefits Abound

CSGs are used to transfer data, including backup images, to a storage-as-a-service provider. CSGs can run on physical or virtual machines and hold data on the LAN until it is time to transfer to the cloud. Users access data through the LAN connection, instead of the slower WAN.

Speed is one of the big benefits to using a CSG together with an image-based backup solution. There are cost and convenience benefits as well – provided the backup solution has the features and functionality needed to optimize CSG use.

So, what are the benefits of using a cloud storage gateway with image-level backups? I see four major ones:

1. When you go to the cloud, you might be able to leave your tape behind. Using a CSG allows you to keep your backups off site and recover them locally, even at the image or file level. Cloud CSGs also provide a DR option, because most of them can be installed in another location, and allow you to connect to your existing data in the cloud. Once you have a CSG set up and working with your image-level backups and you can restore images or files from this configuration, it might be time to think twice about tapes.
2. Adding distance doesn't add time. Even when the images are moved off site to your cloud storage provider, file-level recovery will be nearly instant. Because your image-based backup solution should only request the blocks needed to recover the files you're asking for (instead of the complete backup job), and because nowadays downloads are faster than uploads, your recovery process should be nearly instantaneous. Think about how this compares to trying to locate a tape that was moved off site to fill the same request -- recovery could be days instead of minutes.
3. Only pay for what you need to recover. Since image-based backup can restore files and images from a single image, you're going to save lots of bandwidth on cloud transfer and storage costs. Traditional backup products store files and images side-by-side, which results in more to recover and takes more of your money for bandwidth and storage.
4. Time pressure is reduced. CSGs work on the principal that they have a local cache to allow data to come into the cloud storage gateway as fast as possible over the LAN before it's sent off to the cloud storage provider via the WAN. This is great because you're not compromising your image-based backup window to get your backup data into cloud storage.

  When I talk about the benefits of using the two technologies together I'm assuming that the VM backup software has the following capabilities that support efficient operations:

• The ability to skip deleted data during backup. This is big. I recently did a test and found that skipping deleted data for 35 VMs with only Windows installs came out to a savings of 20 GB. That's 20 GB of data that doesn't need to be sent over the WAN to your cloud storage vendor and 20 GB that doesn't need to be paid for. Some cloud storage provides charge for both data transfer and storage, so skipping deleted data can mean a big savings.
• Turning off compression during backup is also advantageous for minimizing file sizes. Most cloud storage gateways have their own de-duplication and compression features. If you've ever re-zipped a zip file you might have seen it get bigger. The same problem applies when cloud storage gateways de-dupe and compress previously compressed backups.
• Enabling file-level restore from the backup image. If this process is done correctly, the backup software should only request the blocks needed to reassemble the requested files. In a CSG environment, the CSG will only request the necessary blocks from the cloud storage provider – as opposed to requesting the whole archive, whose size might be measured in gigs. File-level restores help keep transfer costs down and allow faster restores.

Like much about the cloud, CSGs are relatively new and subject to change. These are my initial observations and lessons learned about using image-based backup with cloud storage gateways; please share yours.

Posted by Jason Mattox on 02/03/20112 comments

Get More Value From a Cloud Storage Gateway

What is a CSG?
A cloud storage gateway is software that runs on hardware or in a virtual machine that moves data out to one of the many storage-as-a-service providers. CSGs exist to be a holding tank within your LAN until data is moved to your cloud storage vendor. By residing within the LAN, CSGs enable local writes to execute as fast as possible without relying on a slower WAN connection between you and the cloud to write your data.

I have seen a few different approaches to how the local data is dealt with, but in all cases the data is accessible to the user and application even if it is no longer in the local CSG storage. In all cases the software moves the data to the cloud, and when a read request comes in, the CSG will fetch the data to fulfill the read request.

Here is a very simple use case that might help you to see the benefits of using a CSG.

Let's assume that you are not using your great SharePoint site, and instead you're just using well-organized file shares on a CSG. You access your home drive for your documents and pull down the latest version of a Word document that is 320KB. The request is processed through the CSG, which has more than a 1MB connection to the cloud. Because the requested file is only 320KB the request is filled without you even knowing it was pulled down from cloud storage. You can then view, open and edit the file even though it is not in the local LAN.

That's a high-level overview of what CSGs can do. There are several features and capabilities that can make cloud storage gateways more effective. So, what features make good sense with a CSG?

1. De-Dupe Some cloud vendors charge by the byte for sending data and storing it. The more data you put in the cloud, the more you pay. Deduplication capabilities that are built into a cloud storage gateway can significantly reduce the amount of data to be transferred and stored. So if your CSG can reduce your data footprint by 5x for example, that represents a five-fold reduction of your cloud storage bill.

2. Bandwidth control This is very important because users might share the Internet connection and CSG for an application like Salesforce.com. You want to make sure that the CSG does not cripple the Internet connection and prevent users from accessing Salesforce.com or other essential systems during normal working hours. Bandwidth control is a key feature for preserving access and reliability.

3. Bandwidth control on a schedule It's great that I have bandwidth control and I won't cripple my Internet connection, but at night when my users are not in the office I want to open up the bandwidth for the CSG. For example, for the hours of 7 a.m. to 7 p.m., I would want to reserve 1 MB of my 4 MB connection usable by the CSG.

This allows 3 MB of my Internet connect for my users. From 7 p.m. to 7 a.m., during off hours, I would want 3.5 MB of my 4 MB connection available to allow the CSG to get as much data to the cloud as fast as possible while my users are not around. It should be easy for you to schedule bandwidth adjustments and change them as necessary.

4. Being able to delete from the cloud Some vendors cannot remove data from the cloud that you delete locally on the CSG. These vendors have workarounds available and are looking to solve this problem soon. Until they do, not being able to effectively remove data from the cloud will cost you more money for storage, so the ability to delete from the cloud is a money-saving feature.

5. Being able to run as a VM Being able to just toss up a VM for this function makes sense because you're able to utilize your hypervisor's built-in redundancy and your SAN storage for the CSG. Instead of needing a new piece of hardware in your data center for your CSG, let's use what you have.  

Stay tuned! Next, I'll talk about image-based backups and using a cloud storage gateway.

Posted by Jason Mattox on 01/14/20116 comments

Using P2V to Enhance DR Operations

Disaster recovery has traditionally been one of the most popular use cases for virtualization. Now the idea of physical-to-virtual conversion for DR is getting some traction. I agree this is a good idea, and in fact many of our customers use this approach.

First, here's a quick overview of the basic process and requirements:

• The organization needs to have a hypervisor in place to accommodate the physical server after it is virtualized.
• Using physical-to-virtual (P2V) conversion software, a virtual version of the physical server is created.

Posted by Jason Mattox on 12/29/20101 comments