Inside the strange side hustle of teaching AI to think like a lawyer

As the sun sets and the day’s disputes are settled, arbitrator Jessica Crutcher transitions into a different role—training artificial intelligence models. This unique side hustle, facilitated by the company Mercor, involves legal experts like Crutcher crafting complex legal scenarios that challenge AI systems to think critically. From drafting lawsuits related to oil and gas disputes to developing asylum cases, her work consists of prompting the AI and analyzing its responses. In a legal landscape increasingly influenced by technology, many attorneys are recognizing the importance of contributing to AI training. Thousands of legal professionals, including retired judges and paralegals, are dedicating their evenings and weekends to developing intricate legal puzzles, grading AI chatbot responses, and imparting their expert knowledge to these evolving systems. The initiative not only provides insight into the significant technological advancements occurring in the legal field but also allows them to influence the development of tools that could redefine their roles. With over a billion users relying on chatbots like ChatGPT and Claude for assistance with legal inquiries, the demand for precise and reliable outputs is surging. Law firms are turning to AI training as a solution to enhance the quality of these systems, leveraging the professional judgment that only human experts can provide. For Crutcher, the motivation is not solely financial; it’s about staying connected to a rapidly transforming industry. Investors have infused billions into startups aiming to streamline legal processes, akin to how coding tools have revolutionized programming. Crutcher acknowledges that engaging with AI is vital for her ongoing relevance in the profession. Harrison Margolin, another legal expert, shares this sentiment. He began training AI models with Micro1 after observing that his own firm prohibited the use of AI for client matters. Seizing the opportunity, Margolin sought to push the limits of these systems, often creating challenging scenarios that the AI struggled to navigate. One notable task involved constructing a complex cross-border merger with multiple conflicts. Margolin devised fictitious legal documents and a grading system, continuously increasing the difficulty until the AI could not keep up. This process underscores a significant hurdle in legal AI development: unlike programming, where models can learn from vast amounts of public code, legal reasoning is more nuanced and often concealed behind paywalls. Recently, Margolin has shifted his focus to generating 'golden responses'—ideal answers for models to learn from, a process described by Isabel Yishu Yang of Micro1 as converting lawyers into educators. While the compensation for such work is decent, it pales in comparison to what lawyers earn in traditional firms. Despite this, many legal professionals prioritize gaining insight into the future of their industry over monetary gain. Entertainment lawyer Charley Kelsey faced similar challenges while working at Lionsgate, where he was restricted from using AI due to client regulations. Opting to start his practice, he began taking on projects through Micro1, which profoundly transformed his legal approach. One project involved creating a fictional video game and exploring copyright and trademark issues, prompting Kelsey to reassess his instinctual responses to legal questions. As Kelsey delved into the intricacies of the law, he discovered nuances he hadn’t previously considered, such as the copyright eligibility of taxidermy as a potential sculpture. Eventually, his side hustle became his primary occupation when he joined Micro1 full-time. He believes that while AI can expedite tasks like drafting contracts or summarizing research, it cannot replicate the human touch necessary for client interactions or courtroom advocacy. Instead, he views AI as an opportunity to eliminate less nuanced work, allowing lawyers to focus on the complexities that require significant judgment.