Why soaring memory prices could be Nvidia's next big advantage

As memory prices continue to climb, the cost of AI infrastructure is set to rise significantly, potentially impacting Nvidia’s major clients. Recently, Meta's CEO Mark Zuckerberg and Microsoft’s CFO Amy Hood highlighted increased memory expenses as a crucial factor behind their elevated AI investments. This surge in memory costs has sparked a boom in memory stock valuations. Research firm Bernstein noted that the effects could become even more pronounced with the rollout of Nvidia's next-generation AI systems. The firm anticipates that the upcoming Vera Rubin NVL72 system, essential for training and running cutting-edge AI models, could incur costs around $9.1 million per rack, with memory expenses accounting for a substantial portion of this figure. According to Bernstein, prices for high-bandwidth memory are expected to more than triple by the time the Rubin systems are deployed at scale. This could push memory and storage expenses to an estimated $3.2 million per rack, representing over a third of the total expense. Consequently, Nvidia’s major customers could be facing significantly higher bills. Bernstein estimates that constructing a 1-gigawatt data center using the Vera Rubin system will cost approximately $47 billion, up from about $40.5 billion using its predecessor. The report also indicates that costs for networking, cooling, and power delivery are on the rise, further contributing to overall price increases. In light of strong demand for its AI systems and limited competition, Bernstein believes Nvidia has the leverage to transfer these higher memory costs to its customers. "We suspect Nvidia likely employs some form of dynamic pricing strategy and will pass this increase onto customers rather than absorbing it into their margins," the firm stated. Nvidia CEO Jensen Huang has previously cautioned that memory shortages might persist for several years. This report coincides with Nvidia’s recent multiyear partnership with South Korean semiconductor manufacturer SK Hynix, aimed at developing next-generation memory for future systems, including the Vera Rubin. Notably, SK Hynix achieved a market capitalization of $1 trillion in late May, signaling its significant role in the semiconductor industry. Nvidia has yet to respond to inquiries from Business Insider regarding these developments.