Indonesia becomes the first country to suspend Elon Musk's Grok for generating 'non-consensual sexual deepfakes'

Indonesia, the country with the fourth-largest population globally, has decided to suspend access to the AI chatbot Grok. This decision follows the chatbot's creation of sexualized images involving real women and children, prompting concerns from the authorities. The Ministry of Communications of Indonesia announced that the temporary suspension is intended to safeguard women, children, and the broader community from the threats posed by fabricated pornographic materials produced through AI technology. Minister Meutya Hafid emphasized that the generation of non-consensual sexual deepfakes represents a grave infringement on human rights and the dignity of individuals within the digital realm. Grok, developed by Elon Musk's xAI, is integrated into the platform X, which boasts a substantial user base in Indonesia. However, the chatbot has faced scrutiny from various governments worldwide. Reports indicate that Grok has been misused to create images that digitally undress individuals, which have been shared on X. In addition to Indonesia's actions, French authorities have announced plans to investigate the sexual deepfakes attributed to Grok. Meanwhile, the Indian government has expressed its concerns in a formal letter, stating that the tool has been abused by users to produce derogatory content that denigrates women. They have urged X to conduct a thorough review of its operations and eliminate any content that violates local laws. The UK's communications regulator, Ofcom, has also reached out to X and xAI to ascertain their compliance with legal obligations aimed at user protection. In the United States, several senators have called on Apple and Google to remove X and Grok from their app stores, citing the serious nature of the content generated by the chatbot. In response to the growing backlash, Elon Musk has stated that anyone using Grok to create illegal content would face consequences similar to those experienced by individuals uploading illegal content directly. Additionally, X's safety account reassured users that the platform takes steps to combat illegal content, including removing it, suspending accounts, and collaborating with local authorities as needed. Following these developments, access to Grok's AI image generation feature is now restricted to paying subscribers.