In-Depth

Virtualization Takes Wing at Patrick Air Force Base

• By Rutrell Yasin
• 03/31/2010

Replicating data from physical servers to a virtual environment is never simple, but the task is even more daunting with a large, distributed network that holds terabytes of data.

That's the challenge information technology managers at the Air Force's 45th Space Wing at Patrick Air Force Base faced after deciding that virtualization technology was a good way to achieve more efficient IT operations and better use of resources, most of which were running well under capacity. And on top of the sheer size of the job, they wanted to make that migration with almost zero downtime for users.

After a couple of tries, they managed to make a smooth transition using IP-based replication of data.

The wing had in place a storage-area network that had vastly improved the sharing of storage, backup and recovery of data. But as users added more applications and insisted on having their own hardware, the wing wound up with many servers connected to the SAN.

And when IT administrators took a step back to evaluate the situation, they found a lot of servers running nowhere near their full capacity.

"Very few of the servers were running more than 5 to 6 percent of load even during the busy times," said Glenn Exline, manager of enterprise networks at Computer Sciences Raytheon, which supports the 45th Space Wing and Cape Canaveral Air Force Station.

Virtualization seemed like a logical way to get better use out of servers. However, the challenge was how to migrate massive amounts of data from physical servers to a virtual environment.

"We were dealing with some servers that had a terabyte of storage behind them," Exline said. In that type of situation, making sure that users' data is successfully migrated, backed up and restored to a virtual environment can be a daunting task, he said

IT managers evaluated several technologies and settled on VMware's P2V Assistant, a product that has worked well in smaller server environments. However, the volume of data on the servers and storage devices in the wing's environment didn't meet the command's migration needs, Exline said. VMware now offers VMware vCenter Converter for physical-to-virtual server migration.

During tests of P2V Assistant, there were several instances in which IT personnel would schedule time in the evening, from 6 p.m. to 11 p.m, to do P2V migration. The product would take point-in-time snapshots of the server's configuration, so it was best to schedule the conversion of data outside normal usage hours, Exline noted. However, the tool would run for four or five hours, get to 97 percent and then fail — a wasted effort that would put stress on the staff.

The product did not work well when converting data from Microsoft 2000 servers. And servers had to be shut down for hours to convert data. Another approach was to deploy new virtual machines and replicate the data using hardware-specific SAN replication. But that turned out to be too expensive and was not supported on older storage hardware.

The IT team looked at other ways to approach the problem and turned to IP replication, in which data is replicated over IP networks, because it appeared that approach would be easy to implement and replication could be done on the fly. IP replication allows the transfer of data regardless of make, models or processors of the hardware equipment. After evaluating three products, the wing chose CA XOsoft Replication.

The team created a virtual environment and a standard server template based on the latest operating systems and patches. It tested the product using an existing physical server and aligned a virtual server with the older box. Next, it used CA XOsoft Replication to replicate all data and shares from the old server to a new virtual server. After XOsoft indicated that the source and destination data was replicated and in sync, an IT operator would schedule about five minutes of downtime, Exline said.

The team then changed the name of the virtual server to the physical server. The physical server was shut down, and the virtual server rebooted.

"The server was put back online with all the users' data without all of the older associated hardware and software drivers from the physical box," Exline said. The replication of data didn't drag any of the baggage of the older server and upgrades along with it.

"So we had a nice, clean, pristine environment with user data in place, and it cost only about five minutes of downtime," he said.

As a result of using IP replication software, IT managers for the 45th Space Wing completed the physical-to-virtual migration of 60 servers in less than three months. The wing reduced 60 physical servers to four. About 76 virtual machines are running on those four servers.

The Air Force unit completed the migration without any data loss and minimized the impact to users by limiting downtime to less than 10 minutes per server. The wing has also achieved about $200,000 savings annually by virtualizing its IT environment.

The process was not flawless, but it worked well most of the time, Exline said. IP replication could be applied to application servers such as Microsoft Exchange, SQL Server, SharePoint and Internet Information Server, but it does require more planning for certain applications.