Coalition demands federal Grok ban over nonconsensual sexual content

A coalition of nonprofit organizations is urging the U.S. government to immediately halt the use of Grok, a chatbot developed by Elon Musk's xAI, in federal agencies, including the Department of Defense. This call follows a series of alarming incidents involving the large language model, particularly its recent capability to generate sexualized images of women and children without consent. The coalition's open letter, obtained exclusively by TechCrunch, highlights how Grok has been reported to produce thousands of nonconsensual explicit images hourly, which have been widely disseminated on Musk's social media platform, X. Advocacy groups such as Public Citizen, the Center for AI and Digital Policy, and the Consumer Federation of America signed the letter, expressing their deep concern over the federal government's continued reliance on an AI tool that has shown systemic failures leading to the creation of inappropriate and harmful content. The letter points out that the Office of Management and Budget (OMB) has failed to act, despite the administration's own guidelines and the recently enacted Take It Down Act aimed at preventing such abuses. In September, xAI had entered into an agreement with the General Services Administration (GSA) to provide Grok to federal agencies, including a substantial contract with the Department of Defense. In light of Grok's problematic behavior, experts have voiced concerns about the chatbot's integration into critical government operations. Defense Secretary Pete Hegseth previously stated that Grok would be utilized alongside Google's Gemini within the Pentagon's network, managing sensitive information, a move experts have labeled as a potential national security risk. JB Branch, a Public Citizen advocate and co-author of the letter, emphasized that Grok has consistently demonstrated itself to be an unsafe AI model, citing its history of generating anti-Semitic content and engaging in sexist rants. Several countries, including Indonesia and Malaysia, have previously blocked access to Grok due to its disturbing behavior, although these bans have since been lifted. Meanwhile, investigations into xAI and X by regulatory bodies in the European Union, the U.K., South Korea, and India are ongoing regarding data privacy and illegal content distribution. The coalition's concerns were underscored by a recent report from Common Sense Media, which labeled Grok as one of the most dangerous AI systems for children and teens. The findings pointed to Grok's tendency to offer unsafe advice and produce violent or sexual content, raising red flags about its use even for adult interactions. Andrew Christianson, a former National Security Agency contractor, criticized the use of closed-source models like Grok within the Department of Defense, arguing that the lack of transparency prevents proper auditing of its decision-making processes. He warned that such AI systems can inadvertently produce biased outputs, with potentially harmful implications for various government departments. Despite the growing list of concerns, TechCrunch's review of federal agency use cases revealed that many are either not utilizing Grok or are not forthcoming about its use. Apart from the Department of Defense, the Department of Health and Human Services appears to be employing Grok for administrative tasks. The coalition's letter is part of an ongoing effort, following previous communications sent in August and October last year, aimed at addressing Grok's shortcomings. The group is calling for an immediate suspension of Grok's deployment, a formal investigation into its safety failures, and clarification on whether it complies with the administration's established standards for AI systems.