Inside the unseen operation to turbocharge Claude Code

In a significant move to enhance AI coding tools, Anthropic is harnessing the expertise of approximately 1,000 human software engineers through a project referred to internally as 'Marlin.' This initiative aims to refine Claude Code, an AI tool that has recently made waves in the coding sector, striving to replicate the finesse of professional developers. The collaboration with Snorkel AI, a data-labeling platform, illustrates how AI firms often delegate data-related tasks to specialized third parties. These contractors are engaged in a variety of functions, from teaching AI specific subjects to improving model performance. Insights from contractor interviews reveal that they receive $280 per task, with each assignment typically taking around an hour, although some require more interaction with Snorkel’s approval process. Marlin's freelancers, who possess backgrounds in software engineering, were tasked with A/B testing code generated by different AI models. Their evaluations help determine which outputs align best with project specifications. One contractor noted that the project's objective is to train Claude Code to produce code that is easier to maintain and understand. As the field of AI evolves, the focus of data-labeling platforms is shifting towards more specialized tasks necessitating advanced knowledge and degrees. Snorkel AI emphasizes its collaboration with top-tier experts, indicating that some contractors can earn upwards of $3,000 per week. This trend reflects a broader industry movement towards specialization in software engineering training. Contractors working on Project Marlin were asked to devise scenarios for the practical application of Claude Code. They selected GitHub repositories from an extensive list and were responsible for creating Pull Requests, which propose changes to existing codebases. Additionally, they crafted prompts to guide the AI in executing tasks. For instance, one task involved reorganizing the handling of execution metadata to enhance clarity for developers, while maintaining the functionality of the code. Contractors also focused on security enhancements, such as addressing vulnerabilities in the MLFlow platform, ensuring that the proposed solutions blocked command injection attempts while allowing legitimate operations. Founded in 2019 by Stanford researchers, Snorkel AI has become a key player in the AI model improvement landscape, collaborating with leading tech firms like Google and Anthropic. The company recently secured $100 million in Series D funding, achieving a valuation of $1.3 billion. However, it has also seen workforce reductions, indicating the volatility in the tech startup environment. Snorkel is part of a growing ecosystem of startups that hire numerous contractors globally to refine AI responses, contributing to advancements in various domains, including autonomous vehicles and AI-driven chatbots.