Inside OpenAI’s quest to make AI do anything for you

In 2022, Hunter Lightman joined OpenAI as a researcher, just as his colleagues were launching ChatGPT, a product that quickly gained immense popularity. While the spotlight was on ChatGPT, Lightman worked on a specialized team focused on enhancing the ability of AI models to tackle high school mathematics competitions. This initiative, known as MathGen, has become a cornerstone of OpenAI's efforts to develop advanced reasoning models, which are crucial for creating AI agents capable of performing tasks similar to humans. Lightman reflected on the early days of MathGen, stating, “We aimed to improve the models' mathematical reasoning, which was lacking at that time.” Although OpenAI's systems are still not flawless—showing tendencies to generate incorrect information and facing challenges with complex tasks—the progress in mathematical reasoning has been remarkable. Notably, one of OpenAI's models recently clinched a gold medal at the International Math Olympiad, a prestigious competition featuring the brightest high school students worldwide. OpenAI envisions that these enhanced reasoning capabilities will extend to various subjects, ultimately leading to the creation of general-purpose AI agents the company has long aspired to build. The launch of ChatGPT was somewhat serendipitous—a research project that unexpectedly evolved into a viral consumer phenomenon. In contrast, the development of AI agents represents a calculated effort that has been years in the making. Sam Altman, CEO of OpenAI, expressed this vision at the company’s first developer conference in 2023, stating, “In the future, you’ll simply ask a computer for what you need, and it will handle a myriad of tasks for you.” The release of OpenAI's first reasoning model, dubbed o1, in late 2024 sent shockwaves through the tech community. The foundational team behind this innovation quickly became some of the most sought-after talents in Silicon Valley. Major companies, including Meta, aggressively recruited these researchers, sometimes offering compensation packages exceeding $100 million. Shengjia Zhao, one such researcher, was appointed as chief scientist of Meta Superintelligence Labs. The rise of OpenAI’s reasoning models can be attributed to a machine learning technique known as reinforcement learning (RL), which provides AI models with feedback on their performance in simulated environments. RL has been a fundamental aspect of AI development for years, with significant milestones like Google DeepMind's AlphaGo, which famously defeated a world champion in the game of Go in 2016. OpenAI's journey with RL began around the same time, as early employee Andrej Karpathy contemplated its application for developing an AI agent capable of using a computer. It wasn't until 2023 that OpenAI achieved a breakthrough by integrating RL with large language models (LLMs) and a technique called test-time computation. This combination allowed models to engage in more effective problem-solving by verifying their steps before arriving at an answer. This innovative approach, termed “chain-of-thought” (CoT), significantly enhanced the models' performance on previously unseen math problems. Lightman described the moment of discovery as one of the most exciting of his research career, recognizing the potential of these reasoning models to improve AI systems. Following the success of the Strawberry breakthrough, OpenAI formed a dedicated “Agents” team, aiming to further advance this new paradigm of AI capabilities. Initially, the distinction between reasoning models and agents was not clear within the company, but the focus remained on enabling AI systems to tackle complex tasks. Throughout its history, OpenAI has strategically allocated resources, including talent and computing power, to foster innovation. Researchers had to demonstrate breakthroughs to secure the necessary support for their projects. Lightman commented, “OpenAI operates on a bottom-up approach in research, and when we demonstrated the potential of o1, the company embraced it.” This commitment to developing advanced AI models rather than merely products has allowed OpenAI to prioritize projects like o1, a focus that may not have been feasible for competing AI labs. As the field of AI evolves, many researchers acknowledge the need for further understanding of reasoning models. Current AI agents excel in well-defined domains, such as coding, but struggle with subjective tasks. Lightman noted that training models on less verifiable tasks remains a significant challenge in machine learning. OpenAI's researchers are optimistic about their new RL techniques, which have the potential to enhance AI capabilities in various areas, including mathematics. Looking ahead, OpenAI is preparing for the launch of its GPT-5 model, seeking to assert its dominance in the AI landscape. The goal is to create AI agents that intuitively comprehend user needs without requiring specific instructions. This vision represents a significant evolution from the current iteration of ChatGPT, aiming to build an agent capable of performing virtually any task online and understanding user preferences seamlessly. As OpenAI navigates the competitive AI market, the challenge lies not only in delivering on its ambitious vision but also in doing so ahead of formidable competitors like Google, Anthropic, xAI, and Meta. The future of AI is unfolding, and OpenAI's journey is at the forefront of this transformative era.