Intel wants to compete with Nvidia H100 with Gaudi 3 accelerators

With its new Gaudi 3 accelerators, Intel wants to compete with the Nvidia H100, currently the most powerful card for training AI models. According to Intel, it can sell a kit consisting of eight cards and a baseboard for two-thirds of what an Nvidia charges.

The processor designer has its Gaudi 3 cards produced on TSMC's N5 node. They contain 64 of Intel's tensor cores and 8 matrix engines. The package consisting of chiplets also includes 128GB of HBM2e memory with a bandwidth of 3.7TB/s and 96MB of SRAM cache at 12.8TB/s. For connectivity, Intel uses 16 PCIe 5.0 lanes and 24 200GbE ports, which the company emphasizes are part of an open standard, unlike Nvidia's InfiniBand.

According to the maker, a cluster of 8192 Gaudi 3 accelerators is 40 percent faster at training a GPT3-175B model than a comparable cluster of Nvidia H100s. With 64 accelerators and the Llama2-70B model, Intel's advantage would be 15 percent. Depending on the parameters chosen and the AI ​​model used, inferencing should be twice as fast on Gaudi 3 on average.

A Gaudi 3 kit consisting of 8 accelerators and a baseboard has a suggested retail price of $125,000. According to Intel, this would offer a 130 percent better price-performance ratio than a comparable Nvidia H100 kit. In addition, Intel emphasized its good software support during the announcement, where it collaborates with well-known model makers to provide good support for new AI models on 'day zero'.

Dell, HPE, Lenovo and Supermicro already sold systems with the older Gaudi accelerators from Intel, but now ASUS, Foxconn, Gigabyte, Inventec, Quanta and Wistron are added. This increases the number of Gaudi 'integrators' to ten suppliers. Intel's Gaudi branch stems from the acquisition of the originally Israeli Habana Labs in 2019.