The Virtual Test Lab

It makes sense in terms of two emerging trends -- virtualization and Software as a Service.

• By Peter Varhol
• 03/01/2008

A good, comprehensive test lab is one of the most difficult and expensive development facilities to build and maintain. You typically need a lot of systems, several different network options and many images that can be easily copied to a variety of different targets. It also requires a lot of effort to create, store and manage images, run tests, analyze results and track bugs, as well as to administer the systems and network.

I once visited the headquarters of a large insurance company that had a test lab spanning three acres of floor space, along with thousands of server and client images. It included a staff of a dozen just for administration, and testing went on around the clock. Most organizations can't keep that kind of infrastructure simply for application testing.

Many enterprises already spend a great deal on testing. Functional-test, performance-test and test-management software is among the most expensive software in the application development lifecycle. Depending on the size of your team, you could be spending anywhere from tens to hundreds of thousands of dollars on automated testing products.

Further, the way most organizations perform testing today makes for confusion and missed opportunities. Developers write code, and testers test it on various configurations, according to predefined scripts. If the testers find a problem, they describe it in terms of what they did to reach that point, and the effect the problem had on the application. What they don't know is what caused the problem, and where in the code the problem manifested itself.

As a result, developers don't have a lot to go on, and frequently complain that they can't replicate the problem. At best, it may take some time to isolate the problem in code, as well as some time back and forth with QA to determine that it was, in fact, fixed.

Yet testing is critically important for the success of an application. Getting the configurations you need built, software installed, tests run and data gathered and evaluated is a big and complex process.

Virtual Host
One new approach that promises to make application testing more efficient is the virtual testing lab. Such a solution is offered by a company called Surgient (www.surgient.com), which provides development organizations with a complete virtual environment for testing. You still have to do the work, but Surgient provides the infrastructure.

They call it the Virtual QA/Test Lab Management System, or VQMS.

With VQMS, IT and lab administrators can consolidate, pool and centrally manage test infrastructure and configurations that are shared across multiple dev/test teams according to policies and your internal SLAs. Surgient provides the tools to build the lab to your specifications, quickly refresh or add other configurations when needed, and load your software and run your tests. And Surgient cleans up afterward. By virtualizing the test lab and making it available to multiple clients on an on-demand basis, Surgient also takes advantage of its ability to fully utilize its hardware.

Development lifecycle professionals have a difficult enough time building applications today. Some components are built specifically for the application, while others are open source, existing code or proprietary libraries. Once such an application is built, it has to be tested on a variety of operating systems and in conjunction with other applications. Even within an enterprise where server and desktop configurations are strictly controlled, there are dozens of different combinations that should be tested before rolling out the application.

Automated Builds
Perhaps the real value of a virtual testing solution such as Surgient is that it forces development and testing teams to automate their processes. When you're not sitting in front of the computer feeding it DVDs, you can be productive doing other things. I've known some testers who became extremely proficient with Solitaire as they waited for software to load and builds to complete.

Instead, a virtual testing lab forces teams to work with images, scripts, automated error reporting and repeatable processes. It's a short jump from that to automation of builds to performing regular or even continuous builds. Signing up for a virtual test lab may be just the ticket for stepping up your overall processes.

A virtual test lab makes a great deal of sense in terms of two emerging trends -- virtualization and Software as a Service. In this case, rather than software, it's testing infrastructure as a service, but the analogy holds. In the near future, look for more opportunities to leverage your internal development lifecycle resources through online and virtual services. Taking advantage of them may have indirect as well as direct benefits to your application lifecycle.