Upstart chipmakers keep challenging Nvidia. Microsoft-backed D-Matrix is next

In the rapidly evolving landscape of AI chip technology, a new contender has emerged that aims to rival Nvidia, the industry's giant. D-Matrix, a startup located just a stone's throw from Nvidia's headquarters in Silicon Valley, asserts that its latest chip, Corsair, can outperform Nvidia's GPUs in inference workloads by a significant margin. Specifically, D-Matrix claims its chip operates ten times faster while consuming five times less energy, but this advantage holds primarily for smaller workloads. The Corsair chip introduces a groundbreaking memory architecture akin to those developed by Cerebras and Groq. As major tech companies scramble for more computational power, it is becoming increasingly evident that there is ample room for smaller firms to carve out their niches in the market. Following a successful IPO that raised over $5.5 billion, Cerebras is now valued at more than $50 billion. Meanwhile, Nvidia's acquisition of Groq's assets for $20 billion last December highlights its strategy to bolster its capabilities. D-Matrix co-founder and CEO Sid Sheth expressed optimism about the future, stating, "This is a $1 trillion market in the making," and he has no intention of pursuing a sale. Founded in 2019, D-Matrix has secured around $500 million in funding, leading to a current valuation of approximately $2 billion, with Microsoft being one of its investors through the M12 venture arm. While specific clients for Corsair have not been disclosed, Sheth mentioned that the company has garnered interest from several prominent tech companies, hyperscalers, and AI labs eager to enhance their computing resources. D-Matrix plans to initiate shipping to these clients this month, with 90% of them located in the United States and a few in the Middle East and Southeast Asia. Industry analyst Stacy Rasgon from Bernstein Research noted that D-Matrix's customers often utilize its chips alongside Nvidia's offerings, indicating a complementary relationship between different types of chips, which excel at various tasks. The Corsair chip's design allows for low-latency inference on reduced power by integrating memory and computation on a single chip. Unlike traditional GPUs that rely heavily on DRAM, which is currently in short supply, D-Matrix utilizes SRAM, a memory type that can be integrated directly on the chip. However, the reliance on SRAM does present challenges. According to Rick Bahr, an electrical engineering professor at Stanford, while SRAM can achieve impressive inference speeds due to minimal data travel distances, it struggles with the vast reasoning models required for many current AI applications. Sheth counters this by emphasizing that Corsair is engineered specifically for AI inference tasks where speed is critical, such as chatbots and voice assistants. When combined with Nvidia's Blackwell GPU, research indicates that Corsair can outperform standalone GPUs significantly in terms of speed and efficiency. D-Matrix markets its Corsair chips in a configuration where four chips come packaged together, designed for easy installation into server racks at data centers. This plug-and-play model sets D-Matrix apart from competitors like Cerebras and Groq. The Corsair solution is positioned as the most compact SRAM option available today, offering up to 128 gigabytes of SRAM memory. Additionally, D-Matrix has collaborated with Arista, Broadcom, and Super Micro to create the SquadRack, a comprehensive system for deploying their chips in AI-focused data centers. The Corsair chip is produced in Taiwan using TSMC's 6-nanometer technology, with plans for the next-generation Raptor chip to debut next year using TSMC's even more advanced 4-nanometer process. Sheth believes that developing robust AI inference solutions is the ultimate goal in this competitive field.