5 takeaways from CNBC’s investigation into 'nudify' apps and sites

In the summer of 2024, a troubling incident unfolded in Minneapolis when a group of women discovered that a male acquaintance had manipulated their Facebook photos using artificial intelligence to generate explicit deepfake images and videos. Utilizing an AI platform known as DeepSwap, he created nonconsensual depictions of not only his friends but also more than 80 women across the Twin Cities area. The emotional fallout from this revelation was profound, prompting the group to enlist the support of a compassionate state senator. As highlighted by a recent investigation from CNBC, the proliferation of 'nudify' applications and websites has made it alarmingly easy for individuals to create and share nonconsensual explicit content. Experts warn that these services are widespread online, often marketed through Facebook advertisements, available in both the Apple and Google app stores, and easily discovered through basic web searches. According to Haley McNamara, senior vice president at the National Center on Sexual Exploitation, the current state of technology poses a significant risk, making anyone potentially vulnerable to such exploitation. The investigation sheds light on the complex legal landscape surrounding AI-generated content and the plight of these women as they seek justice. In a situation rife with challenges, the absence of criminal charges against the perpetrator—due to the lack of underage victims and non-distribution of the deepfakes—has been particularly frustrating. Molly Kelley, one of the victims and a law student, expressed concern over the legal loopholes that allowed the man to escape accountability: "He did not break any laws that we're aware of, and that is problematic." In response, Kelley and her peers are advocating for legislative reform in Minnesota, pushing for a bill introduced by Democratic state Senator Erin Maye Quade. This proposed law aims to prohibit nudify services within the state and impose hefty fines on those facilitating the creation of such deepfakes. Maye Quade likened the bill to existing laws that prevent unauthorized photography, emphasizing the need for legal frameworks to keep pace with rapid AI advancements. The psychological toll on the victims has been severe. Jessica Guistolise, another affected woman, described ongoing struggles with anxiety and panic, particularly triggered by the sound of a camera shutter—a reminder of the traumatic experience. "I heard that camera click, and I was quite literally in the darkest corners of the internet," she recounted, reflecting the profound impact of being victimized in such a manner. Mary Anne Franks, a professor at George Washington University Law School and president of the Cyber Civil Rights Initiative, compared the trauma of these experiences to that of revenge porn victims, stating, "It makes you feel like you don't own your own body, that you'll never be able to take back your own identity." While deepfake technology once required advanced skills to manipulate, the emergence of user-friendly nudifier applications has lowered the barrier for entry, allowing anyone with internet access and a photo to partake in this disturbing trend. Despite disclaimers about consent on many nudify sites, enforcement remains ambiguous, and the marketing of these applications often disguises their true intent. DeepSwap, the site involved in the creation of these deepfakes, has raised questions regarding its transparency and operational legitimacy. Recent developments indicate that the company has changed its name and jurisdiction multiple times, further complicating efforts to hold it accountable. Senator Maye Quade's proposed legislation seeks to impose fines of $500,000 on tech companies for every nonconsensual deepfake produced in Minnesota. However, some experts express concern that federal initiatives to enhance the AI sector may undermine state-level efforts to regulate these harmful applications. With the Trump administration prioritizing AI development as a national security concern, advocates like Kelley fear that their fight for justice may be overshadowed by broader geopolitical ambitions. "I'm concerned that we will continue to be left behind and sacrificed at the altar of trying to have some geopolitical race for powerful AI," she cautioned.