Yes, AMD has a secret weapon to fight off Nvidia AI armada — no, it has absolutely nothing to do with GPUs and everything to do with HBM

The AI arms race is heating up with AMD leaning on innovations by Xilinx and SK Hynix

When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.

AMDwill rely on advancements in high-bandwidth memory (HBM) in its bid to unseatNvidiaas the industry leader for making the components that power generative AI systems.

Building on the theme of processor-in-memory (PIM), Xilinx, which is owned by AMD, showcased its Virtex XCVU7P card, in which each FPGA had eight accelerator-in-memory (AiM) modules. The firm showcased this at OCP Summit 2023, alongside SK Hynix’s HBM3E memory unit, according toServe the Home.

Essentially, by performing compute operations directly in memory, data won’t need to move between components on systems, meaning performance increases and the overall system becomes more energy efficient. Using PIM, with SK Hynix’s AiM, led to ten times shorter server latency, five times lower energy consumption, and half the costs in AI inference workloads.

The latest twist in the ongoing AI arms race

Nvidia and AMD make most of thebest GPUsbetween them, and one may assume that efforts to improve the quality of these components are key to improving AI performance. But it’s actually by tinkering with the relationship between compute and memory do these firms see there are huge advantages to be made in power and efficiency.

Nvidia is also racing ahead with its own plans to incorporate HBM technology into its line of GPUs, including the A100, H100 and GH200, which are among thebest graphics cardsout there. It struck a deal withSamsunglast month for incorporate its HBM3 memory technology into its GPUs, for example, and will likely extend this to include the new HBM3e units.

PIM has been something several companies have pursued in recent months. Samsung, for example, showcased its processing-near-memory (PNM) in September. TheCXL-PNMmodule is a 512GB card with up to 1.1TB/s of bandwidth.

This follows a prototype for an HBM-PIM card, which was made in collaboration with AMD. The addition of such a card boosted performance by 2.6% while boosting energy efficiency by 2.7% against existing GPU accelerators.

Are you a pro? Subscribe to our newsletter

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

More from TechRadar Pro

Keumars Afifi-Sabet is the Technology Editor for Live Science. He has written for a variety of publications including ITPro, The Week Digital and ComputerActive. He has worked as a technology journalist for more than five years, having previously held the role of features editor with ITPro. In his previous role, he oversaw the commissioning and publishing of long form in areas including AI, cyber security, cloud computing and digital transformation.

Phishing your own people in cybersecurity training? How to protect the brands you use as bait

Cisco issues patch to fix serious flaw allowing possible industrial systems takeover

Lenovo ThinkPad T14s Gen 6 review