Microsoft announces powerful new chip for AI inference

Microsoft has introduced its latest innovation, the Maia 200 chip, which is specifically engineered to enhance AI inference capabilities. This new silicon powerhouse succeeds the Maia 100, launched in 2023, and is designed to execute advanced AI models more quickly and efficiently. The Maia 200 is a remarkable feat of engineering, featuring over 100 billion transistors that enable it to achieve more than 10 petaflops of performance at 4-bit precision and about 5 petaflops at 8-bit precision. This performance boost marks a significant advancement over its predecessor. Inference involves the execution of AI models, a process that has become critical as companies seek to optimize their operational costs. Microsoft aims for the Maia 200 to play a crucial role in streamlining AI operations, reducing the need for extensive power consumption and minimizing disruptions. The company emphasizes that a single Maia 200 node is capable of managing the largest AI models available today, with ample capacity for even more complex models in the future. The launch of Maia 200 reflects a growing trend among technology giants to develop proprietary chips, reducing reliance on NVIDIA’s GPUs, which have become essential for AI advancements. Similar initiatives include Google’s tensor processing units (TPUs), which are offered through cloud services rather than as standalone chips, and Amazon’s Trainium accelerator, which recently released its Trainium3 version. In a competitive landscape, Microsoft claims that the Maia 200 outperforms third-generation Amazon Trainium chips by three times in FP4 performance, and exceeds the performance of Google’s seventh-generation TPU in FP8. Currently, the Maia chip is already powering various AI models from Microsoft’s Superintelligence team and is integral to its Copilot chatbot. The company has also extended an invitation to developers, researchers, and AI labs to experiment with its Maia 200 software development kit, encouraging collaboration in pushing the boundaries of AI technology.