How-To

ARM in the Datacenter

Tom Fenton highlights some different ARM devices and discusses ways that they have made their way into the datacenter.

• By Tom Fenton
• 04/19/2021

ARM processors have seen unprecedented growth in the last three years and are now being used in everything from smart watches to Apple's new M1 desktop and laptop systems, but there is one sector where they have yet to take hold: the enterprise market. For years, many of the largest cloud providers have designed computers around ARM chips, and in December 2020, Microsoft said it was joining the fray by designing its own ARM-based chips for Azure and Surface PCs. Now we are seeing technology based on ARM chips float down from the cloud providers and rise up from the consumer market and start to take hold in the datacenter. In this article, I will highlight some different ARM devices and discuss ways that they have made their way into the datacenter.

Datacenter ARM Server
All the major server suppliers have either announced, or are already selling, ARM servers. We are also seeing companies that are solely focused on Arm-based servers.

An example of the type of system that established server companies are selling is the Lenovo ThinkSystem HR330A, a 2U rack server designed for datacenter usage. It has a 32-core 64-bit ARMv8 CPU running at 3.3 GHz, and supports 512GB of RAM. For storage, it supports two NVMe drives in the chassis, and six external storAge bays.

One of the premier companies focused on ARM servers is Ampere Computing. It was founded in 2017 by former Intel president Renée James to create ARM servers for the datacenter. Its high-end model is a 2U rack server with dual Altra Q80-33 processors. Each processor has 80 cores running at up to 3.3GHz, and has 16 DIMM slots allowing up to 4TB DRAM capacity per processor, or 8TB in the server.

Bamboo Systems is another company that specializes in ARM servers. The B100N ARM server is another good example of the compute power that can be achieved using ARM processors. It is composed of blades that are made up of four compute nodes that each have 16-core ARM A72 Cortex processors. Two blades are placed in a 1U system for a total of 128 ARM cores.

Supercomputers
In the top 500 supercomputer rankings list published in June of 2020, Japan's Fugaku ARM-based supercomputer took the number one spot. It performed 416 petaflop/s on the High Performance Linpack (HPL) benchmark, beating the previous year's non-ARM based supercomputer by 280 percent. It has 158,976 Fujitsu A64FX processors running at 2.2GHz. Each processer contains 48 ARM cores for a total of slightly more than 7.5 million cores.

Storage
Although I haven't seen any ARM-based storage arrays from the major vendors yet, many smaller vendors for SMB and ROBO storage arrays use ARM processors.

SoftIron leverages ARM's lower cost, power efficiency, and SoC architecture to power their storge arrays. They use Ceph as the software for their device as it is a mature product (it was first released in 2012), and has been extensively developed by major commercial Linux distributions such as Ubuntu, Red Hat, and SUSE, as well as Intel, Cisco, and SanDisk.

Their family of arrays range from a 48TB (8 x 6TB HDD) server with a single 10GbE interface, to a 112TB SSD (28 x 4TB SSD) system with dual 100GbE interfaces.

Computational Storage
Computational storage is a new term that I am starting to see emerge to describe the ability to process data directly on the same device as the storage. Neil Werdmuller, ARM's Director of Storage Solutions, described computational storage as "... a critical piece of the data storage puzzle because it puts processing power directly on the storage device, giving companies secure, quick and easy access to vital information." The processing can be done directly from the storage OS, or by using virtual machines (VMs) or containers that also run on the storage appliance.

In a traditional datacenter, both physical and virtual machines connect to a storage array to process the data.

By having the hypervisor on the same physical system as the storage, the network latency is removed, but the storage can also be used by other servers.

ARM-Based Computational Storage
Computational storage appliances marry storage and VMs on the same physical server. This signifies that storage can be effectively delivered using ARM processors. Two examples of storage arrays being powered by ARM processors are SoftIron and QNAP.

SoftIron enterprise storage systems, as mentioned above, not only provide storage, but they also can perform computational activities as well, such as transcoding data.

QNAP makes a range of network attached storage (NAS) appliances, ranging from a single disk appliance for home use to their 24-Bay Active-Active Dual Controller unit designed for larger environments. QNAP uses both x64 and ARM processors in their NAS appliances.

The TS-1635AX NAS uses an ARM processor and supply storage, as well as containers and VMs. It uses Linux to manage the VMs and containers. QNAP has a catalog of different applications that can run directly on their storage appliances.

HCI
Sunlight is a company that makes a full-stack hyperconverged infrastructure (HCI) solution (SDS, hypervisor, networking) that has been highly optimized to take advantage of ARM processors and NVMe drives. It will be interesting to watch this company and see if they can become a player in the highly-competitive HCI market.

SmartNIC
SmartNIC is an interesting technology that allows for the processing of data on the NIC rather than on the computer, freeing up the server to do more productive work. Many of these SmartNICs use ARM processors, and some even use a hypervisor and VMs for the applications that process the data.

Software
Software compiled for x64 systems will not run on ARM systems. Fortunately, all the major Linux distributions and many hypervisors have ARM versions.

Microsoft announced Windows 10 on ARM in December 2016, and has support for some ARM devices. However, it is unlikely that you can get it to run on a generic ARM system. Microsoft has announced that it does have an ARM-based server running Windows on its cloud, but it is unlikely that we will see a version of it available to the public in the near future.

Ubuntu, Red Hat, Fedora and SUSE are just a few of the Linux distributions that have ARM distributions. However, due to the difference in hardware design, not all these distributions will run on ARM hardware, so a little research is advised before attempting to install Linux on an ARM system. An interesting side note is that Japan's Fugaku Arm-based supercomputer runs Red Hat.

On the hypervisor side, VMware has released ESXi on ARM as an unsupported Fling. This has gotten a lot of attention and supports a wide range of hardware, from a Raspberry Pi to 32-core servers. XenServer has an ARM version and KVM is included with many of the Linux for ARM distros. Of course, you will need to run ARM guest VMs on these systems.

Conclusion
ARM processors are starting to become more and more ubiquitous in the datacenter, not only for servers but also for infrastructure. For instance, we have ARM-based SmartNICs that offload work from servers, and we also have appliances that not only provide the prerequisite NAS and SAN storage, but also do compute functions as well. On the software side, all the major Linux distributions and hypervisors run on servers with ARM processors. Although ARM devices have a strong foothold in the datacenter due to their low cost, high performance, and energy efficiency, I only envision that we will see more and more ARM processors in the datacenter in the future.