Apple, Google pull ‘nudify’ apps after probe finds platforms profited from deepfake abuse

In response to a recent investigation, Apple and Google have started to eliminate AI-driven 'nudify' applications from their app stores. This decision follows findings that these platforms were inadvertently profiting from tools designed to create non-consensual sexual imagery. The Tech Transparency Project (TTP) revealed that over 100 applications capable of digitally 'undressing' individuals were available to millions of users, despite existing policies from both companies prohibiting sexually explicit content. A separate investigation by CNBC highlighted the plight of over 80 women in Minnesota, who were victimized when their publicly shared social media photos were utilized by these nudify services to generate sexualized deepfakes without their consent. The TTP report indicated that advancements in AI technology have simplified the creation of explicit deepfake content, which is now easily accessible through consumer-friendly applications. Among the reviewed apps, 14 were based in China, raising concerns over data retention laws that could allow the Chinese government access to sensitive data generated by these services. Following the release of the TTP report on January 27, Apple confirmed to CNBC that it had removed 28 of the identified apps and issued warnings to other developers. Google also took action, initially suspending several apps and later removing 31 from its Play Store as part of a broader review process. The TTP estimates that these applications collectively garnered over 705 million downloads and generated around $117 million in revenue. Given that both companies typically retain a commission of up to 30% on in-app purchases, researchers argue that they were indirectly profiting from this abusive content. This controversy has also sparked renewed scrutiny of Elon Musk’s Grok chatbot, which is integrated into the social media platform X. TTP researchers noted that a search for 'nudify' in Apple’s App Store prominently featured Grok as the leading organic result. Additionally, Copyleaks estimated that Grok was producing approximately one non-consensual sexualized image per minute during the peak of this scandal earlier this month. In light of mounting global pressure, including a cease-and-desist letter from California’s attorney general and threats of a ban in the UK, xAI has restricted Grok’s image-editing capabilities to paying users and has implemented geoblocking in certain regions. The TTP report criticized both Apple and Google for not keeping up with what it termed a 'mass digital undressing spree' facilitated by generative AI. Many of the flagged apps had passed standard review processes and were deemed appropriate for children as young as four or nine. Michelle Kuppersmith, the executive director of TTP, stated, 'Apple and Google should be vetting the apps in their stores. However, they’ve allowed numerous apps that can be used to depict individuals with minimal or no clothing, making them vulnerable to misuse.' The findings have ignited calls from U.S. senators and civil society organizations for an outright ban on applications that enable technology-facilitated sexual abuse, as regulators in the UK, Europe, and India increase their scrutiny of online platform safety practices.