AI’s next phase to drive higher computing demand, not less, warns Deloitte

According to a recent report by Deloitte, the artificial intelligence sector is on the brink of a significant increase in its demand for computing power. This shift comes as businesses transition from merely constructing AI models to implementing them on a large scale. The consulting firm's annual Technology, Media and Telecommunications outlook, released on March 18, challenges the notion that the rise of inference—running AI models for query responses—will lessen the reliance on costly chips and expansive data centers. Deloitte predicts that by 2026, inference will represent approximately two-thirds of AI computing, a marked increase from one-third in 2023. This growing reliance on inference will not alleviate the financial burden on infrastructure, as the firm emphasizes the necessity for all planned data centers and enterprise-grade AI facilities, along with the substantial energy these operations will require. The report estimates that global investment in AI data centers could soar to around $400 billion by 2026, potentially escalating to as high as $1 trillion annually by 2028. The persistent demand for computing resources is attributed to two developing strategies: post-training scaling and test-time scaling. These methods significantly amplify computing needs, with post-training techniques, like fine-tuning and reinforcement learning, consuming about 30 times the resources needed for the initial model training. Meanwhile, test-time scaling can demand over 100 times the computing power for a basic inference task. Many AI companies are now adopting these resource-intensive techniques to enhance their models. Deloitte also notes that while the market for inference-optimized chips could exceed $50 billion by 2026, this will not diminish the need for high-performance graphics processing units (GPUs) essential for AI training. The report describes the chip market as evolving to support both categories simultaneously, predicting that high-end chips, which can exceed costs of $30,000 each, will account for roughly $200 billion in spending by 2026. The total AI chip market is expected to surpass $400 billion by 2028. Despite advances in chip efficiency, Deloitte reports that the demand for AI computing is growing at a remarkable pace, estimated at four to five times each year, a trend projected to persist through 2030, with significant implications for global energy consumption. The role of edge devices, such as smartphones and PCs, in handling advanced AI tasks is expected to remain limited. Most AI computation by 2026 will occur in large-scale AI data centers or on high-end AI servers. In India, Deloitte anticipates a rapid expansion of data center capacity, which is expected to increase from about 1.5 gigawatts in 2025 to around 10 gigawatts by 2030, fueled by rising AI demands and the adoption of 5G technologies. Power consumption from these data centers could reach approximately 57 terawatt-hours by March 2030, raising the sector's share of national electricity usage to between 2.5% and 3%. While there is speculation that future breakthroughs could lead to decreased computing needs, Deloitte believes that such advancements are unlikely to materialize by 2026.