The Reinforcement Gap — or why some AI skills improve faster than others

Recent advancements in AI coding tools have been remarkable, with innovations like GPT-5 and Gemini 2.5 introducing new capabilities for developers. These enhancements are not always apparent to those outside the coding world, but they have significantly improved automation in software development. For example, last week saw the launch of Sonnet 2.4, which further pushed the boundaries of what AI can achieve in coding. However, not all AI skills are evolving at the same pace. Take email writing, for instance; users might find their experience remains largely unchanged from a year ago, even as underlying models improve. This inconsistency in AI development can often be traced back to the nature of the tasks involved. Tasks that can be easily quantified and graded, like coding, benefit from reinforcement learning (RL) — a powerful technique that has driven much of the progress in AI over the past six months. Reinforcement learning thrives on clear metrics that allow for repeated testing, making it ideal for tasks that can be objectively evaluated, such as bug fixing or mathematical challenges. In contrast, skills like creative writing or chatbot interactions, which are inherently subjective, do not see the same level of advancement. This disparity has led to what many are calling a 'reinforcement gap' in the AI landscape, influencing the capabilities of various AI systems. Software development is particularly suited for RL applications, given its history of rigorous testing protocols to ensure code reliability. Developers routinely conduct unit tests and integration tests to validate their work, and these same processes can be adapted to assess AI-generated code effectively. As Google’s senior director for developer tools noted, these established testing methods are invaluable for reinforcing AI systems as well. However, not all tasks fit neatly into the categories of easy or difficult to test. For example, while financial reports may seem complex, a well-funded startup could potentially create an effective testing framework. The ability to measure a process plays a crucial role in determining whether it can be transformed into a viable product. Interestingly, some areas previously considered challenging to evaluate are proving to be more testable than anticipated. Take AI-generated video, for instance; OpenAI's latest Sora 2 model demonstrates significant advancements, showcasing stable object recognition and realistic physics. These improvements suggest that robust reinforcement learning systems are likely at play, bridging the gap between mere visual appeal and true photorealism. This dynamic is not set in stone. As AI models continue to evolve, the role of reinforcement learning may shift, potentially altering the reinforcement gap. But for now, as RL remains a primary driver of AI product development, the divide in capabilities will likely widen, impacting startups and the broader economy. For instance, the extent to which healthcare services can be trained with RL could shape job markets and industries over the next two decades. With breakthroughs like the Sora 2 model, the answers may come sooner than we expect.