When AMD launched its Instinct MI300X accelerators for AI and HPC about a year ago, Amazon Web Services (AWS) expressed interest in deploying them in the cloud. However, according to Amazon, as reported Business Insiderdue to lack of strong demand, the company has still not done so.
“We’re following customer demand,” Gadi Hutt, director of product and customer engineering at Annapurna Labs, an Amazon company, told Business Insider. “If customers have strong indications that these elements are needed, there is no reason not to deploy them.”
At least, according to Hutt, there hasn’t been enough interest to justify deploying AMD’s Instinct MI300X accelerators at AWS. While AMD’s Instinct MI300X is cheaper than Nvidia’s H100, its software is not as robust as Nvidia’s CUDA, which scares many developers. As AMD’s hardware offerings improve (e.g. Instinct MI325X), so should its software.
To some extent, Hutt can be considered an interested party, because Trainium developed in Annapurna competes with those of AMD and Nvidia in AWS data centers. However, assuming he has spoken on the record, this is AWS’s position.
Speaking of Trainium, with its in-house designed Trainium and Trainium2, AWS doesn’t have to pay extra to AMD or Nvidia, which is why it can offer Trn1 and Trn2 instances at very competitive prices compared to powered ones. by Nvidia’s H100 GPUs. . This may be another reason for the low interest in third-party, non-Nvidia solutions.
Speaking of Nvidia, AWS announced at its re:Invent conference that it is set to bolster its AI offerings with Nvidia’s upcoming Blackwell GPUs for AI and HPC. At the re:Invent conference, AWS showed off its P6 servers powered by Blackwell GPUs, reflecting the expectation that these machines will be in high demand.
Although it doesn’t offer AMD’s Instinct MI300X in the cloud, AWS continues to work closely with the company and offers many instances based on AMD’s EPYC processors. Given their core count and memory subsystem, these processors offer huge advantages over their Intel Xeon rivals for compute- and memory-intensive instances.