Videos of sexually suggestive, AI-generated children are racking up millions of likes on TikTok, study finds

A recent study by an online safety non-profit has revealed a troubling trend on TikTok, where AI-generated videos depicting underage girls in suggestive attire have garnered millions of likes. Despite the platform's strict policies against such content, over a dozen accounts showcasing these alarming videos have amassed a substantial following. The accounts feature AI-generated images of girls clad in provocative clothing, including lingerie and school uniforms, often posed in inappropriate ways. The report, led by Carlos Hernández-Echevarría from Maldita.es, highlights that many videos include comments linking to Telegram chats, where illicit content is allegedly sold. Even after TikTok's reporting tools flagged 15 offending accounts last week, 13 of them remained active as of Wednesday evening. This raises significant questions about TikTok's capability to enforce its own guidelines when it comes to AI-generated material that sexualizes children. As jurisdictions worldwide, including Australia with its new social media restrictions for users under 16, intensify their focus on online safety, the pressure mounts on platforms like TikTok to ensure the protection of younger audiences. Hernández-Echevarría emphasized the straightforward nature of the issue, stating, "There is absolutely no way a human being sees this and doesn’t understand what’s happening." TikTok has publicly declared a zero-tolerance policy on content that promotes or depicts the sexual exploitation of minors, explicitly forbidding accounts that share AI-generated images of children in adult-like scenarios. The company claims to employ a blend of automated and human moderation to manage its content, stating that a significant percentage of violating content is proactively removed. The findings from Maldita.es were discovered through test accounts used to monitor harmful content trends. The report noted that some accounts described their videos in a manner that would attract inappropriate attention, referring to AI-generated images as “delicious-looking high school girls.” The trend even extends to seemingly innocuous videos, such as young girls eating ice cream, which are met with crude comments. The concern is further amplified by the interaction between these TikTok accounts and Telegram, where researchers found links leading to sites selling AI-generated videos depicting minors in a sexualized manner. Although researchers did not engage in any transactions, the information has been reported to law enforcement in Spain. In response to the alarming nature of these findings, Telegram stated its commitment to preventing child sexual abuse material and noted extensive measures taken to eliminate such content from its platform. Meanwhile, TikTok continues to face scrutiny over its content moderation practices, especially following multiple reports of inappropriate suggestions directed at young users. The implications of these findings are significant as they highlight a critical gap in safeguarding young users on social media platforms, prompting discussions on the need for stricter regulations and more effective enforcement of existing policies.