Google unveils chips for AI training and inference in latest shot at Nvidia

In a bold move to compete with Nvidia in the AI hardware arena, Google has announced a significant shift in its chip design strategy. The tech giant revealed that it will be splitting the functions of training artificial intelligence models and handling inference tasks into two specialized processors. This change comes with the introduction of its eighth generation tensor processing unit (TPU), set to debut later this year. Amin Vahdat, Google’s senior vice president and chief technologist for AI and infrastructure, emphasized the necessity of this change, stating that the AI ecosystem would benefit from chips tailored specifically for training and serving. This decision aligns with the growing complexity of AI agents, which require more precise and efficient processing capabilities. Nvidia, a major player in the AI chip market, has also been making headlines with its upcoming silicon designed for rapid user interactions, leveraging technology acquired in a significant deal with startup Groq. While Google remains a substantial customer of Nvidia, its TPUs serve as an alternative for businesses leveraging its cloud solutions. The competition among tech giants to develop custom semiconductors for AI is intensifying. Companies like Apple, Microsoft, and Meta are also investing heavily in specialized AI chips. For instance, Apple has integrated neural engine components into its iPhone chips, while Microsoft introduced its second-generation AI chip earlier this year. Meta is collaborating with Broadcom to create various AI processors, showcasing the industry's shift towards in-house chip development. Google was an early adopter of AI-centric processors, having commenced their use in 2015 and offering them to cloud clients starting in 2018. Analysts from DA Davidson recently valued Google’s TPU business, in conjunction with its DeepMind AI group, at approximately $900 billion. However, despite these advancements, no tech company has succeeded in overtaking Nvidia in the market. Google’s new training chip reportedly delivers 2.8 times the performance of the previous generation Ironwood TPU while maintaining the same cost. The inference processor also shows an impressive 80% performance improvement. Notably, both chips utilize static random-access memory (SRAM), with the new inference chip, TPU 8i, containing 384 MB of SRAM—three times the capacity of its predecessor. According to Sundar Pichai, CEO of Google’s parent company Alphabet, the new architecture is optimized for high throughput and low latency, enabling the simultaneous operation of millions of AI agents cost-effectively. Adoption of Google’s AI chips is gaining momentum, with significant organizations such as Citadel Securities and all 17 U.S. Energy Department national laboratories leveraging the technology for advanced research applications. Additionally, Anthropic has pledged substantial resources to utilize Google’s TPUs.