Thinking Machines Lab wants to make AI models more consistent

Mira Murati's Thinking Machines Lab, backed by an impressive $2 billion in seed funding and a powerhouse team of ex-OpenAI researchers, is making waves in the AI field. Recently, the lab unveiled one of its ambitious projects aimed at enhancing the consistency of AI models in a blog post titled "Defeating Nondeterminism in LLM Inference." This initiative seeks to unravel the underlying factors responsible for the randomness often observed in AI responses. When users pose the same question to ChatGPT multiple times, they frequently receive varied answers. This phenomenon has been widely acknowledged within the AI community, where current models are deemed non-deterministic. However, the team at Thinking Machines Lab believes this challenge is not insurmountable. The launch of their research blog, Connectionism, marks a significant step in their journey to address this issue. In the inaugural post, researcher Horace He delves into the intricacies of AI randomness, pinpointing the orchestration of GPU kernels—tiny programs that operate within Nvidia’s chips—as a primary source of unpredictability during inference. He argues that by meticulously managing this orchestration layer, it may be possible to achieve more deterministic AI models. The implications of creating AI that delivers consistent outputs extend beyond mere reliability for businesses and scientists; it could also streamline reinforcement learning (RL) processes. Without the noise generated by inconsistent responses, the RL training regimen could become much more efficient, enhancing AI performance. Thinking Machines Lab has signaled its intention to leverage RL for tailoring AI models to meet specific business needs, as previously reported by The Information. In July, Murati, who formerly served as OpenAI's chief technology officer, hinted that a product from the lab would be introduced soon, designed to assist researchers and startups in developing bespoke models. While specific details about the product remain under wraps, there is speculation that it may incorporate insights from this research to foster more reliable outputs. The lab's commitment to share insights through regular blog posts and code releases reflects a desire not only to advance public knowledge but also to enrich its own research culture. This first installment in the Connectionism series offers a rare insight into one of Silicon Valley's more enigmatic AI startups. While it does not disclose the exact trajectory of their technology, it signals that Thinking Machines Lab is confronting some of the most pressing questions in AI research. The ultimate challenge will be whether the lab can effectively address these issues and create marketable products that validate its hefty $12 billion valuation.