After the recentAmpere Altra Launch where we found the next-gen Arm CPU will have 80 cores, we realized a challenge. This was a chip destined to be shown at OCP Summit 2020 along with the two partner platforms. With the OCP Summit going virtual, we needed a way to show the new chips and platforms without the exhibit hall. I did what most would do in my shoes, and just drove 15 minutes down the road to Ampere HQ Santa Clara to see the new Ampere Altra Q80-30 parts in-person.
Ampere Altra Q80-30 Hardware
Press photos did not do the Ampere Altra justice. This is one big chip! Here is the Altra for size comparison next to a Diet co*ke 12oz can. I had just come from theHPE-Cray El Capitan 2 Exaflop Supercomputer event and needed some afternoon caffeine which provided a nice reference point for size.
The chip itself uses a nice LGA pad array on the bottom. One thing is for certain, this is a much larger package than 2nd Gen Intel Xeon Scalable Refresh chips and larger than the package (minus the carrier) of theAMD EPYC 7002 Series “Rome” CPUs. Our test lab is about 8 minutes away from where we have both x86 chips so I drove there just after my visit to validate that perception. I have handled a few hundred Xeon Scalable and EPYC parts since 2017 so it seemed bigger when I was at Ampere’s office and that was confirmed by checking the Xeon and EPYC parts just after.
Since someone will likely ask, here is the side view of the chip.
I know that many will want better chip shots, however, this was a smartphone plus conference room lit session rather than in our Sunnyvale photography/ video studio.
Ampere Altra 2U Servers
During the announcement, we saw two servers that are sampling now.One “mystery” server seemed like it may be a Wiwynn server given Wiwynn’s supporting quote in Ampere’s announcement, even though the drive trays looked a bit different on the Mt. Jade platform. The other was much easier to see it is a Gigabyte server similar to one STH has tested before.
During the visit, we were able to see both servers in their OCP Summit 2020 display booth.
Ampere Mt. Jade
Here is a look inside the 2U dual-socket Mt. Jade platform. As one can see, this platform is wired for 24x front panel NVMe bays along with two processors. Each processor has 16 DIMMs for a total of 32 DIMMs. One of the first things I noticed was the room between the outermost DDR4-3200 DIMM slots and the side of the chassis was smaller than on most AMD EPYC 7002 series servers. Wiwynn needed to use high-density cabling to make this work.
The CPU here had its markings easier to read. One can see the Ampere Altra Q80-30 is there. Knowing what we do about the product, it seems like the “80” may correspond to 80 cores and 30 to 3.0GHz but that is speculation. If that is the case, and Ampere indeed releases the 3.3GHz part they used in their EPYC 7742 comparison, we would expect that SKU to be something like the Altra Q80-33.
With the Ampere Altra Q80-30 socket, one will notice that there are a total of five screws that need to be secured whereas the AMD EPYC 7002, also with a top latching mechanism, uses four. Intel Xeon Scalable attaches with a bracket to the cooling solution then has four screws and two pins and is significantly different.
Inside the system, we can see a few other features aside from the Wiwynn branding. There is an NVIDIA Tesla T4. We covered NVIDIA CUDA on Arm Becoming a Reality and now Arm server vendors are highlighting support. We have heard that recently, perhaps due to the first three Exascale supercomputer losses for NVIDIA, the company has started to work more aggressively on its NVIDIA CUDA for Arm project.
Aside from the GPU, we can see two internal M.2 slots and an OCP 3.0 NIC that presumably will support PCIe Gen4 in this platform since that is one of the goals with the new form factor. One will also note an absence of a PCH like Intel Xeon CPUs currently have. That PCH is not a feature many hyper-scalers are overly fond of. As we would expect, there is an Aspeed BMC onboard for management functions. If you are not familiar with Wiwynn, they make servers for large hyper-scale companies such as Microsoft and Facebook. I had the opportunity to see Where Cloud Servers Come From Visiting Wiwynn in Taipei.
Ampere Mt. Snow by Gigabyte
The second system Ampere calls “Mt. Snow”. This platform is a Gigabyte server that looked very familiar. In-person, we saw the Gigabyte MP32-AR0 motherboard in a chassis that looked familiar. Indeed, Gigabyte essentially replaced the AMD EPYC 7002 motherboard in the Gigabyte R272-Z32 with the MP32-AR0 motherboard and it now has an Arm server.
This means the system is powering all of the 24x front panel NVMe slots from the Ampere CPU’s PCIe lanes. We even see a similar layout with the OCP 2.0 NIC slot being utilized for U.2 connectivity. Here is a somewhat similar view from the Gigabyte R272-Z32 platform:
As one can see, it appears as though the Ampere socket and DIMMs are slightly wider on the motherboard than the AMD EPYC 7002 series despite each having 16 DDR4-3200 DIMM slots. It is not by much, but a good clue is near the CPU power connectors on the edge of the board one can see an Aptio BIOS sticker between the power connectors and the DIMMs. There is no room for this common square sticker on the Ampere system.
Still, this is a good sign that Gigabyte can put the Ampere solution into an existing AMD EPYC 7002 series platform with a straight-forward motherboard change.
Ampere Altra Q80-30 Software
One of the more interesting facets of my visit was getting to see the new 80-core Arm CPU running OpenStack and Kubernetes.
As you can see, the company had an image library with OSes such as CentOS, Debian, Fedora, FreeBSD, and openSUSE.
The company also had a Kubernetes in VMs cluster running using k3OS.
The physical machine was running in a (loud) conference room next to where we were meeting. I was able to see the system running OpenStack. It sounded like the recipe for OpenStack on a single Ampere Altra node was relatively new, but it is an important step for being able to show that the new CPUs work in an existing ecosystem.
We did not get to benchmark the system. It will likely be a few months until that is possible. Realistically, the “launch” that happened a few days ago was more of an announcement to the market rather than full volume production of parts and systems. When the platforms are ready, we will bring you numbers.
Final Words
My sense is that Ampere, at this point, is not where Intel usually is when it “launches” a CPU. Today’s Intel launches happen well after hyper-scale clients have been getting datacenter wings full of servers, and hundreds of thousands of other “early access” servers have been sold. The recent Ampere launch was really to let the market know that Ampere has silicon, and if you want to get started on it, you should start preparing. For STH readers who do not work at hyper-scale clients, my general sense is that we are still waiting until around mid-2020 until we start seeing some of these platforms become available. Realistically, Ampere needs to jumpstart its platform partner ecosystem as well and those platforms can take months to develop.
Still, this is exciting times in the Arm server chip industry. We are seeing more new entrants challenging the x86 status quo. If anything, since 2017 we have seen competition in the market start to return and that is having a big impact on the pace of innovation which is a win for STH readers. Stay tuned to STH as we will bring you benchmarks and hands-on use impressions as soon as Ampere turns us loose with a box.
