The great memory squeeze: How AI is rewriting the rules of the chip industry

The dominant trio of global memory manufacturers is uniting in a warning that the tech sector cannot afford to overlook. Micron Technology, a key player in the U.S. memory market, recently revealed its decision to withdraw from the Crucial consumer sector to concentrate on the rapidly expanding AI and data center arenas. This announcement was quickly followed by a cautionary note from Samsung Electronics, indicating that a tightening of memory supplies could significantly affect pricing across the broader electronics market. Meanwhile, SK Hynix, despite dismissing speculation about exiting the consumer market, is ramping up its investments in advanced packaging to cater to the soaring demand for high-bandwidth memory (HBM). These developments collectively suggest a critical understanding: the ongoing global memory shortage is a structural issue rather than a temporary cycle, with implications expected to extend well beyond the immediate future. Notably, the repercussions are no longer limited to AI infrastructure but are reshaping the cost dynamics across both enterprise IT and consumer electronics, with price pressures anticipated to last into 2027 or even 2028. Historically, memory types such as Dynamic Random Access Memory (DRAM) and NAND were viewed as commoditized products, subject to the familiar boom-and-bust cycles driven primarily by demand from PCs and smartphones. However, the rise of AI has dramatically altered this landscape. In smartphones, memory is utilized mainly to enhance operating system efficiency and app responsiveness within tight power and thermal constraints. Traditional enterprise servers rely on substantial DRAM pools for databases and virtualization, with consistent throughput needs but seldom requiring extreme bandwidth. In contrast, AI servers have transformed this framework. "Large models, activation data, and continuous parameter updates demand sustained, ultra-high bandwidth to keep massively parallel accelerators engaged," explains Manish Rawat, an associate analyst at TechInsights. "In AI systems, the processing speed can outpace the delivery capacity of conventional memory, making memory the limiting factor rather than compute power." This disparity has revealed the inadequacies of traditional memory hierarchies and propelled the industry towards high-bandwidth memory and advanced architectures that prioritize throughput over cost-per-bit. Consequently, memory has transitioned from a supporting role to a critical performance bottleneck. According to Sanchit Vir Gogia, CEO and chief analyst at Greyhound Research, a single AI training server can house up to 4 terabytes of memory across CPU DRAM and GPU-based HBM, in stark contrast to the 16 GB of RAM typical in high-end laptops and smartphones. "This isn't a minor increase; it represents a fundamental shift," he asserts. The surge in AI demand coincided with memory manufacturers just beginning to recover from one of the sector's most significant downturns. The oversupply witnessed in 2022-2023 led to a drastic price drop, compelling producers to scale back on capital investments and reduce wafer output. When the demand for AI surged unexpectedly, the industry found itself ill-prepared. Despite the heightened demand, memory manufacturers are approaching production scaling with caution. Memory production is one of the most capital-intensive sectors within the semiconductor industry, leading suppliers to avoid the pitfalls of previous over-expansion cycles that resulted in sharp price declines. "After enduring numerous boom-and-bust cycles, manufacturers are now emphasizing disciplined capital expenditure, expanding only in alignment with long-term demand," Rawat highlights. Moreover, suppliers are focusing on profitability over sheer volume. Memory products tailored for AI, particularly HBM and server-grade DRAM, yield significantly higher margins compared to consumer-grade memory. This trend has motivated manufacturers to direct production capacity towards data centers at the expense of smartphones and PCs. "Even with surging demand from AI, cloud, and enterprise sectors, suppliers are not rushing to increase output," Gogia adds, noting that resources are primarily being allocated to higher-yield opportunities like HBM and premium DRAM for data centers, which restricts availability for general-purpose DRAM and consumer-grade NAND. Any new production capacity often becomes locked into long-term contracts with major clients, tightening the supply for the larger market. Consequently, industry experts increasingly anticipate that memory constraints will remain until at least 2027 or 2028. Over the past two years, memory prices have escalated sharply, transitioning from a minor cost factor to a significant pressure point across devices and infrastructure. In smartphones, memory typically constituted about 10-15 percent of the total bill of materials, but by 2025, this figure is expected to exceed 20 percent, particularly in lower-end models. In budget smartphones, memory has now become the most costly component after the system-on-chip. DRAM prices have surged nearly 70 percent, while NAND prices have more than doubled in the last 18 months, as reported by Greyhound Research. This increase in component costs is projected to dampen demand, with global smartphone shipments forecasted to decline by 2.1 percent in 2026, according to Counterpoint Research. The same trend is evident in laptops and notebooks, where memory and storage have become significant cost burdens, accounting for up to a quarter of total system costs. Most devices now come equipped with 16 GB of DDR5 memory and a 512 GB SSD, both of which have experienced significant price hikes. "As a result, enterprises refreshing their fleets are feeling this cost pressure in real time, either having to pay more or compromise on specifications," states Gogia. The situation is even more critical in servers, where memory is now not just an expensive component but increasingly the primary driver of overall system costs. A standard dual-socket server featuring 512 GB of DDR5 can see memory costs represent 30-40 percent of the bill of materials. In AI workloads, GPU accelerators like NVIDIA’s H100, which include 80 GB of HBM each, can see memory account for over half the total cost of the accelerator when factoring in packaging and board-level design. For the time being, AI infrastructure is shouldering the heaviest memory cost burden, with enterprise IT following closely. Consumer markets have managed to remain somewhat insulated through product tiering, with OEMs absorbing costs temporarily. However, as shortages continue, pricing pressure is expected to gradually flow from memory suppliers to system manufacturers and ultimately to consumers.