The Hoard Facts

Blog archive

This Guy Gets It

Dan Shipley is a guy who knows how to get the most out of an IT department. As a data center architect at Supplies Network, a half-billion-dollar wholesale supplier of computer supplies based in St. Louis, he is overseeing a data center refresh that is heavy on virtualization and cloud computing. He is also going to productize and sell the process knowledge he develops during this refresh toward the goal of turning his 20-person (and growing) IT department into a revenue-earning business that treats internal and external customers as if they were dealing with a dedicated consulting firm.

Before joining Supplies Network a year ago, Dan had worked for three previous companies who hired him to build out their virtual infrastructures working predominantly in VMware environments. At those firms, he started virtualizing servers and moved on to applications and desktops.

"I treat our IT department as though it is a consulting business," he says, adding "Even though we are a wholly owned inhouse IT department, we provide the kinds of best practices you would expect from an external provider. We measure the consumption of services and charge back to both internal and external customers."

Dan is a big InfiniBand fan because of its performance, security and Quality of Service (QOS) features, which he feels sets it apart from other fabrics such as Gigabit Ethernet and Fibre Channel over Ethernet (FCoE). Specifically, he says that InfinBand's reliance on the RDMA protocol enables servers to exchange large amounts of data without CPU intervention. [Editor's note: This sentence was corrected.]

He looked at FCoE during a nine-month evaluation period, but had doubts about its performance, long-term costs and immature standards. He says Xsigo virtual I/O offers up to four times the performance of FCoE at a lower cost and in a fully-open environment that eliminates vendor lock-in. He goes on to note that Xsigo's InfiniBand fabric can scale beyond 2,000 nodes and provides performance of up to 40Gb/s per link.

Moving from internal to external considerations, Dan says that developing a cloud platform for Supplies Network required a flexible interconnect strategy that enabled data center resources to be linked when there was a need to meet application demands, but isolated as needed to support data security demands. His cloud platform includes ESX server clusters based on HP rack mount Nehalem servers with 48 gigs of RAM each. He says he will run out of server power before he runs out of bandwidth.

One thing he won't run out of is good management ideas that make his IT department look good.

Question: Would InfiniBand make your virtualization infrastructure better?

Posted by Bruce Hoard on 07/28/2010 at 1:09 PM


Reader Comments:

Fri, Jul 30, 2010 Brian

Just as a point of interest...the encoding for IB is changing from the 8b/10b to 64/66 in the next speed grades of FDR 56Gb/s and EDR 104Gb/s.

Thu, Jul 29, 2010 Michael Domingo Irvine, CA

John, my bad on the RDA/RDMA thing (and I noticed that Dan already chimed in with a comment, but really, it's my bad). I didn't catch this error when it came my way in the editing. It's been noted and corrected in the blog above. Apologies.

Thu, Jul 29, 2010 Dan

John, you are correct that the current Xsigo IB switch is DDR at 20G, but the QDR switch will be out in a few weeks. Most servers have dual connections to redundant Xsigo directors in a HA configuration for 40G total bandwidth. The HCA's in the servers are already QDR, and will automatically run at 40G (80G total per server) with the simple switch swap. You are also correct that RDMA is the direct memory transfer version of protocols used without the need for expensive TOE cards or processor overhead, in particular SRP (SCSI), iSER (iSCSI), and NFSoRDMA (NFS). I would also point out that IB adapters (HCAs) and switches at 40G are cheaper than 10G Ethernet, and much less than 10G FCoE, while also being an open, mature standard with drivers for Windows, Linux, and VMWare.

Thu, Jul 29, 2010 John

There's no such thing as RDA, it's RDMA (Remote Direct Memory Access) for InfiniBand. Also, with the Xsigo system it uses 4X InfiniBand DDR, so 20Gbps (bi-directional.) Either you're trying to do "Cisco math" with the 40Gbps number in the article, or it's wrong based on the performance of the Xsigo system. Of course, those are the signaling rates, not the data rates for InfiniBand. 4X InfiniBand SDR = 8Gbps, DDR = 16Gbps, and QDR = 32Gbps of actual data rates.

Wed, Jul 28, 2010 Jacob

yes, infiniband makes virtual infrastructure better. It is not as expensive as some would have you think... and it helps you to transfer $$$ from energy to better performing per watt technologies sooner.

Add Your Comment:

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above