Anthropic says Chinese hackers jailbroke its AI to automate a 'large-scale' cyberattack

In a startling revelation, Anthropic has disclosed that state-sponsored hackers from China successfully commandeered its AI model, Claude, to orchestrate a significant cyberattack with minimal human input. According to a blog post published by the company, Claude was responsible for executing approximately 80-90% of the cyber intrusion, targeting around 30 organizations worldwide. Anthropic expressed strong confidence that a Chinese state-backed group was behind this operation. The targeted entities included major technology firms, financial institutions, chemical manufacturing companies, and various government agencies. Anthropic noted that the hackers achieved success in infiltrating a limited number of these organizations. AI agents, designed to perform tasks independently, are increasingly utilized by businesses for repetitive operations like customer support, enhancing productivity. However, this same technology can be repurposed for illicit activities. In August, Anthropic reported thwarting an earlier attempt by cybercriminals to exploit Claude for hacking operations, albeit with smaller teams. While AI has been a tool in hacking endeavors for years, the company believes this incident marks the first well-documented case of a large-scale cyberattack predominantly executed by AI. Despite Claude's built-in safeguards intended to prevent misuse, the hackers managed to bypass these protections by fragmenting their requests into smaller parts that did not set off any alerts. They masqueraded as a legitimate cybersecurity firm conducting defensive tests. Utilizing Claude's capabilities, the attackers conducted reconnaissance on the digital infrastructures of their targets and crafted code to breach defenses, enabling them to extract sensitive data such as usernames and passwords. Anthropic has chosen to make its findings public to aid the cybersecurity sector in fortifying defenses against AI-enhanced hacking strategies. The company's blog post emphasized the staggering efficiency of the AI, stating, "The sheer amount of work performed by the AI would have taken vast amounts of time for a human team. The AI made thousands of requests per second — an attack speed that would have been impossible for human hackers to match." While other tech giants like OpenAI and Microsoft have reported nation-state involvement in cyberattacks utilizing AI, these instances primarily involved content generation and code debugging rather than autonomous task execution. Jake Moore, a global cybersecurity advisor at ESET, remarked that the incident is not unexpected, highlighting that automated cyberattacks can outpace human-led efforts and overwhelm traditional defenses. He added, "This situation underscores how these attacks enable low-skilled actors to initiate sophisticated intrusions at relatively low costs." As AI continues to streamline operations for cybercriminals, it also plays a crucial role in defensive strategies. Moore concluded that security now relies on automation and speed, rather than just human expertise, across organizations.