Google's new chips are a shot at Nvidia — and a big hint at where AI goes next

In a bold move to reshape the AI landscape, Google introduced its latest AI chips on Wednesday, marking a significant shift in focus toward both training and inference capabilities. This new generation of Tensor Processing Units (TPUs) aims to position itself as a formidable competitor to Nvidia's established dominance in the AI chip market. The unveiling highlighted two distinct chips: the TPU 8t, designed specifically for training the most advanced AI models, and the TPU 8i, optimized for inference—the critical process of executing models after deployment. Google plans to make both chips available later this year, indicating a strategic response to the evolving demands of the AI industry. As the competition in AI heats up, Google recognizes the need to enhance computing power for applications that leverage AI models. The shift towards inference is becoming increasingly important, with industry leaders focusing on optimizing performance in this area. In response to this trend, Nvidia has also been proactive, securing a $20 billion licensing agreement with Groq, a chipmaker specializing in inference, and recently launching its own chip aimed at enhancing inference speed. Google's new TPU 8i marks a significant advancement from its previous generation, boasting improved high-bandwidth memory (HBM) to address the 'memory wall'—a challenge where processors struggle to access necessary data quickly. This enhancement is crucial for running AI agents effectively. Google Cloud CEO Thomas Kurian described the development of these new chips as a 'natural evolution' of their technology. He emphasized the importance of power efficiency, stating that as demand for training and inference scales, managing energy consumption will be paramount. The evolution of AI capabilities is shifting from basic question-answering to more complex reasoning and actions, according to Google's infrastructure leaders. This shift reflects a broader industry trend, as tech giants like Google, Amazon, and Microsoft race to develop custom silicon solutions to reduce reliance on Nvidia's chips while still utilizing them for various applications. While Google has trained its advanced Gemini models using its own TPUs, it continues to offer Nvidia's chips through its Google Cloud services. The company also plans to provide access to Nvidia's upcoming Vera Rubin GPUs later this year. With over a decade of experience in silicon development, Google's intensified efforts in recent years aim to attract new clients and potentially lessen Nvidia's market grip. Analysts at Morgan Stanley have noted that selling around 500,000 TPU chips could significantly enhance Google's revenue, projecting an increase of approximately $13 billion by 2027.