Anthropic ditches its core safety promise in the middle of an AI red line fight with the Pentagon

Anthropic, a startup established by former OpenAI members concerned about AI risks, has decided to relax its fundamental safety principles in light of increasing competition. The company is moving away from its self-imposed restrictions on AI model development to adopt a more flexible, nonbinding safety framework that it claims can evolve over time. In a blog post released on Tuesday, Anthropic acknowledged that its two-year-old Responsible Scaling Policy might be a barrier to competing effectively in the rapidly expanding AI landscape. This policy shift is particularly notable as it coincides with a tense confrontation with the Pentagon regarding AI safety regulations. The Pentagon recently issued a warning to Anthropic, demanding that the company reconsider its AI safeguards or risk losing a substantial $200 million contract. The ultimatum came during a meeting with Defense Secretary Pete Hegseth, who indicated that noncompliance could lead to Anthropic being placed on a government blacklist. In its announcement, Anthropic explained that its previous safety commitments were intended to foster industry-wide consensus on managing AI risks, but they felt increasingly out of touch with the current political climate in Washington, which favors reduced regulation. The former policy had included provisions for pausing the training of more powerful models if their capabilities exceeded the company's ability to ensure safety— a measure that has been omitted in the new guidelines. Anthropic now argues that responsible developers pausing their progress while others race ahead could lead to a less safe environment overall. The company has also stated that it will decouple its internal safety strategies from its recommendations for the industry at large. The revised safety framework, termed the “Frontier Safety Roadmap,” outlines new public goals rather than stringent commitments, marking a significant shift from its prior approach. This change comes shortly after Hegseth's ultimatum, highlighting the mounting pressure from both government and competing firms. Despite these challenges, Anthropic remains firm in its stance against specific uses of AI, particularly regarding AI-operated weapons and mass surveillance of citizens. The company believes that AI lacks the reliability required for such applications and that there are currently no laws governing its use in surveillance contexts. AI researchers have voiced their support for Anthropic's position on social media, expressing concern over the potential implications of AI in government surveillance. The company, which has built its reputation on prioritizing safety, recently pledged $20 million to Public First Action, an advocacy group promoting AI regulation and education. As Anthropic navigates this complex landscape, it faces not only governmental pressures but also fierce competition from rivals like OpenAI, as both companies strive to roll out new AI tools for enterprise use. Jared Kaplan, Anthropic’s chief science officer, emphasized that the decision to relax their safety commitments was not merely a reaction to competitive pressures, asserting that they did not believe halting AI training would benefit anyone in the long run.