Google Cloud launches two new AI chips to compete with Nvidia

On Wednesday, Google Cloud introduced its latest generation of custom-designed AI chips, known as tensor processing units (TPUs), which have been divided into two distinct models. The TPU 8t is specifically optimized for model training, while the TPU 8i focuses on inference, the process that occurs after users submit prompts to the AI. These new TPUs boast impressive specifications when compared to their predecessors, claiming to deliver up to three times faster AI model training, 80% improved performance per dollar spent, and the capability to connect over a million TPUs in a single cluster. This advancement is expected to provide significantly more computational power while reducing energy consumption and costs for customers. Despite these advancements, Google’s new chips do not represent a direct challenge to Nvidia’s dominance in the market—at least not immediately. Like other major cloud service providers such as Microsoft and Amazon, Google intends to use these chips as a complement to its existing Nvidia-based offerings rather than a complete replacement. Additionally, Google has announced that it will include Nvidia’s latest chip, Vera Rubin, in its cloud services later this year. As the landscape evolves, there is potential for hyperscalers like Google, Microsoft, and Amazon to rely less on Nvidia as enterprises increasingly migrate their AI needs to cloud platforms and adapt their applications for these new chips. However, the current market dynamics suggest it remains unwise to underestimate Nvidia's influence. Notably, chip market analyst Patrick Moore humorously recalled his 2016 prediction that Google’s TPU launch could spell trouble for Nvidia and Intel. Fast forward to today, and Nvidia has become a powerhouse with a market cap nearing $5 trillion, indicating that initial forecasts may not have accurately anticipated the company's trajectory. If Nvidia’s strategy unfolds as planned, Google's expansion as an AI cloud provider could actually lead to increased business for Nvidia rather than a decline, even with many workloads relying on Google's proprietary chips. In a collaborative effort, Google and Nvidia are also working together to enhance computer networking, particularly focusing on improving the software-based networking technology called Falcon, which Google made open-source in 2023 under the Open Compute Project.