RSI is the new AGI — and it’s just as hard to pin down

The term "recursion" is gaining traction in artificial intelligence discussions, with several startups adopting the name and many others integrating Recursive Self-Improvement (RSI) into their strategic plans. Similar to the way Artificial General Intelligence (AGI) has been perceived, RSI has emerged as a shorthand for a potential explosive advancement in AI, though there is still some debate regarding its precise definition. In essence, RSI refers to AI systems capable of perpetually enhancing themselves. Once these systems can manage their own upgrade cycles more efficiently than humans, they could enter a self-sustaining loop, constrained only by their computational resources, potentially rendering human input unnecessary. This concept fascinates many AI laboratories eager to explore its possibilities. Recently, renowned AI researcher Richard Socher unveiled a new venture called Recursive Superintelligence, explicitly aimed at achieving RSI. "Our primary objective is to create a truly recursive, self-improving superintelligence at scale," Socher explained to TechCrunch, highlighting the ambition to automate the entire cycle of ideation, implementation, and validation of research. Other notable researchers are also pursuing this ambitious goal, including Alex Karpathy, a prominent figure known for his work at Tesla and OpenAI. Karpathy's project, dubbed Auto-Research, employs agent swarms to train large language models (LLMs) on straightforward tasks. He has been transparent about the project's progress, sharing updates on social media and providing foundational tools via a public GitHub repository. Although his current efforts have focused on refining models similar to GPT-2, many in the research community have been inspired by his vision of recursive self-improvement. Adaption, a startup founded by Sara Hooker, formerly of Cohere and Google, has introduced AutoScientist, a tool designed to automate frontier training. This system, like Karpathy’s, aims to enable agents to make gradual enhancements, with the ultimate goal of facilitating the training of comprehensive frontier models. If successful, this could propel research capabilities closer to the RSI ideal. Doris Xin, founder of Disarray, drew attention to RSI when her self-trained machine learning agent achieved 28 medals in a recent Kaggle competition, surpassing many human-trained models. Xin argues that the primary obstacle is reliability, suggesting that with unlimited computational resources and time, the goals of RSI could already be within reach. She emphasizes that the pursuit of RSI is less about creativity and more about robust engineering. Despite this enthusiasm, many experts acknowledge that the AI sector is still far from realizing truly recursive systems. Google CEO Sundar Pichai recently remarked that while progress is being made, the concept of RSI represents a significant leap forward that is not imminent. The continuum of self-improving AI is evident, as shown by a lead programmer at Anthropic, who claimed that nearly all of his team's code was generated by their AI tool, Claude Code. This raises questions about whether AI tools could eventually replace engineers altogether. In a recent survey related to the Mythos preview, several Anthropic engineers expressed the belief that advancements could soon allow AI to substitute for a mid-level engineer. However, engineers still recognize significant limitations, particularly in areas requiring self-direction, which is crucial for RSI. The AI industry's timeline for achieving meaningful recursive systems remains uncertain. A study by Georgetown’s Center for Security and Emerging Technology revealed a divide among experts, with some anticipating an impending surge in AI capabilities while others expect more gradual advancements. Helen Toner, director of CSET and a former board member at OpenAI, asserts that simply utilizing AI tools for research does not equate to achieving RSI. She emphasizes that true RSI would involve systems capable of conducting research independently of human intervention. Ayeja Cotra from METR has proposed various milestones on the path to AI research autonomy, suggesting that AI may already be close to the adequacy threshold, where it can produce some output without human input. While the journey toward RSI may seem linear, Toner warns that overcoming the challenges of delegating the entire research process to machines will be complex. Drawing parallels to the evolution of computing, she notes that while humans have gradually relinquished control over various processes, they have always retained some level of oversight. Achieving a fully recursive AI system poses significant engineering and alignment challenges, particularly in balancing human labor with machine intelligence. Ultimately, the consensus among researchers is clear: while the concept of a fully recursive AI system remains a tantalizing vision, it is not yet a reality.