Elon Musk’s Grok ordered to stop creating AI nudes by Dutch court as legal pressure mounts

In a landmark ruling, a Dutch court has prohibited Elon Musk's xAI and its chatbot Grok from producing non-consensual AI-generated images, including those of adults and children. This decision, handed down by Amsterdam's District Court, reflects the increasing legal scrutiny faced by the company. The court's order specifically bans xAI from generating or distributing sexual imagery where individuals are depicted partially or completely nude without their explicit consent. This ruling is unprecedented in Europe and encompasses cases of child sexual abuse material, effectively barring the platform formerly known as Twitter, now called X, from offering Grok's services in the region. Should xAI fail to comply with the ruling, it faces a hefty penalty of €100,000 (approximately $115,000) for each day of non-compliance, with a maximum potential fine reaching €10 million. The case was initiated by Offlimits, a Dutch nonprofit dedicated to combating online sexual abuse, particularly against minors. Additionally, xAI has been ordered to cover Offlimits' legal fees amounting to €2.2 million within the next two weeks. Robbert Hoving, managing director of Offlimits, remarked, "The judge drew a clear line yesterday: technology is not a license to violate human rights online." He also noted the significance of a recent European vote to ban tools that enable the creation of non-consensual nudity, celebrating it as a vital step toward safeguarding online rights across Europe. Despite xAI's previous efforts to restrict Grok from generating sexualized images of real individuals on X, the measures were found to be easily circumvented, prompting the court to deem them inadequate in protecting users. During a recent period, Grok was estimated to have generated around three million sexualized images, with over 23,000 appearing to depict children, according to the Center for Countering Digital Hate (CCDH). In addition to the Dutch ruling, xAI is also facing multiple lawsuits, including one from Baltimore, which alleges violations of consumer protection laws and misleading marketing of Grok and X as safe platforms. The complaint highlighted a troubling trend where Grok was used to alter images of individuals to appear in inappropriate scenarios. Moreover, three teenagers in Tennessee have filed a lawsuit against xAI after Grok produced explicit content portraying them in suggestive poses. Concurrently, the European Commission has opened an investigation into xAI for the dissemination of non-consensual sexual material, as part of its oversight under the Digital Services Act. U.K. regulators are also investigating the platform in response to these serious concerns. While no comprehensive ruling has yet been established throughout Europe, nations like Malaysia and Indonesia have taken action by blocking access to Grok due to the ongoing risks associated with the AI tool.