As HBM shortages bite, AMD turns to software to stretch AI memory

AI infrastructure has long been measured in compute power. But as companies race to train and deploy generative AI models, memory is emerging as an equally critical constraint.

Demand for High-Bandwidth Memory, or HBM, which sits alongside AI accelerators, has surged. Memory makers including Samsung, SK Hynix and Micron are prioritising specialised HBM production over standard memory, tightening supply across the ecosystem. Analysts expect shortages and elevated prices to remain structural features of the market through at least 2027.

Must read: India may need $80 billion in government incentives by 2035 to build semiconductor ecosystem

That pressure is being felt by enterprise buyers trying to scale AI without allowing infrastructure costs to rise at the same pace. It is this friction that AMD is seeking to address through its acquisition of MEXT, a startup focused on memory optimisation.

Making memory go further

MEXT has developed an AI-powered platform designed to make flash storage behave more like DRAM. By optimising how data is stored and retrieved, the technology aims to improve memory efficiency, reduce dependence on expensive DRAM and lower AI infrastructure costs.

Rather than keeping data in premium memory, MEXT predicts what an application is likely to need next and manages how that data moves between memory and storage.  

“Memory has become one of the biggest constraints in AI infrastructure, with HBM shortages limiting both AI training and inference. Software techniques such as memory tiering, caching, and compression can improve utilisation by 20–50%, it can mainly help customers get more value from existing memory resources,” explained Pareekh Jain, chief executive officer at EIIRTrend & Pareekh Consulting.

Optimisation, not a cure

The technology addresses a real problem, but analysts caution that it should not be mistaken for a solution to the global memory shortage.

“MEXT’s predictive memory technology should be seen as a memory optimization solution rather than a replacement for DRAM. While flash storage cannot match DRAM’s latency or bandwidth, intelligent data placement can improve access efficiency and reduce reliance on high-cost memory for certain workloads,” said Manish Rawat, semiconductor analyst at TechInsight.

Must read: AI boom sparks rush for engineers who can speak both code and business

Amid persistent supply constraints, Rawat said the technology could help data-centre operators improve resource utilisation, lower infrastructure costs and reduce procurement pressure. However, it cannot create additional memory capacity or remove the need for premium memory in performance-intensive AI systems.

The distinction matters because the benefits will not be uniform across workloads.

“For instance, MEXT is best suited for databases, analytics, vector search, AI inference, and cloud infrastructure workloads where data access patterns are predictable. Benefits are likely to be smaller for frontier AI training, HPC simulations, and other workloads that require extremely high memory bandwidth and low latency,” added Jain.

This means MEXT may be more useful for enterprises running inference, databases and analytics than for companies training frontier models, where speed and bandwidth requirements leave little room to substitute DRAM or HBM with slower storage.

A gradual integration

AMD believes the acquisition can strengthen its position as a provider of full-stack compute and AI infrastructure. It plans to integrate MEXT’s technology across its data-centre portfolio, potentially extending the benefits to EPYC server processors, Instinct AI accelerators and the software stack supporting those products.

Must read: NVIDIA says its new liquid cooling system can reduce water and energy use for AI data centre

Customers, however, are unlikely to see immediate gains. Integrating memory-management software across a hardware portfolio will probably be a multi-stage process, beginning with software-level improvements before expanding into deeper hardware and software co-design.

“AMD is likely to integrate MEXT first through its software stack, improving memory tiering and infrastructure efficiency across EPYC CPUs, Instinct GPUs, and ROCm. Customers could start seeing benefits within 12–18 months, with deeper hardware-software integration arriving over subsequent product generations,” added Jain.

Can it narrow Nvidia’s lead?

The acquisition comes as competition in AI infrastructure moves beyond raw chip performance. Nvidia’s advantage is built not only on its GPUs, but also on CUDA, networking, mature software tools and an entrenched developer ecosystem.

MEXT gives AMD another way to improve the economics and efficiency of its AI offerings. Analysts, however, view the acquisition as an incremental addition rather than a shift capable of transforming the competitive landscape.

Jain said the deal strengthens AMD’s ability to lower the total cost of ownership and improve infrastructure utilisation, but does not fundamentally close Nvidia’s lead. Nvidia’s advantage continues to come from its full-stack platform, software ecosystem and networking capabilities, making MEXT a useful enhancement rather than a competitive game changer.

Must read: From Mumbai to Jamnagar: India's data centre map is rapidly expanding, shows report

Rawat said better memory utilisation could still deliver meaningful TCO gains for hyperscale deployments as model sizes and HBM costs rise. MEXT, he said, should be viewed as a force multiplier rather than a standalone game changer.

If successfully integrated with AMD’s Instinct accelerator roadmap, ROCm software stack and wider data-centre portfolio, the technology could become a differentiator in next-generation AI infrastructure.

For AMD, the acquisition is ultimately a bet that software can ease one of AI’s pressing infrastructure constraints. For enterprise customers, it offers a route to scaling AI workloads without a proportional increase in memory spending, even if it cannot make the industry’s memory shortage disappear.