Harvey CEO says the company's AI usage jumped from 1 trillion tokens a month to 12 trillion

In an eye-opening revelation, Winston Weinberg, CEO of the legal AI startup Harvey, disclosed that the company’s monthly consumption of AI tokens has surged dramatically. Speaking on the "Sourcery with Molly O'Shea" podcast, Weinberg noted that usage has skyrocketed from 1 trillion tokens in January to an astonishing projected range of 12 to 13 trillion tokens by May. Harvey's AI tools are increasingly relied upon by law firms and corporate legal departments for drafting, reviewing, and analyzing a variety of documents. Weinberg emphasized the rapid growth in token use, stating, "The usage is getting insane." In the context of AI, tokens represent the units of text processed by models as users input prompts and receive responses. The complexity of tasks and the capabilities of the AI models directly influence the number of tokens utilized. As the demand for AI continues to rise, many tech companies are becoming more vigilant about their expenditures on AI services. Reports have emerged indicating that some startups are reevaluating their strategies to avoid excessive token consumption in light of escalating costs. For instance, Coinbase is directing certain prompts to more cost-effective models, while Uber executives are questioning the return on investment from their AI development tools. Weinberg highlighted the emerging challenge for businesses: determining the appropriate level of AI usage for different tasks. He cautioned against employing high-end AI for every function, remarking, "You don't want frontier intelligence running every task. It's too expensive." For example, while a detailed review may justify using advanced AI, a preliminary document summary may not. The CEO drew a parallel between the scrutiny of AI spending and the traditional billing practices of law firms, referring to it as the "billable hours problem." He pointed out that law firms itemize their services in six-minute increments to demonstrate value to clients. As companies face increasing costs associated with token usage, Weinberg warned that they will need to provide clear justifications for their expenditures, asking, "I just spent $1 billion on tokens. Where's my ROI?"