Sora might have a 'pervert' problem on its hands

The new video-making application, Sora 2, developed by OpenAI, has become a source of concern regarding the use of celebrity likenesses, particularly in contexts that some users find disturbing. While the app has been a hit since its launch, allowing users to create videos featuring their favorite personalities, it has also led to the emergence of fetish-themed content that raises ethical questions. A user recounts their experience with Sora 2, noting that after enabling the feature that allows others to use their likeness, they discovered unsettling videos featuring fetishistic themes. Among these, one particular video depicted the user in a scenario that suggested pregnancy, surrounded by an array of similar fetish content on the creator's profile. This user's realization of the app's potential misuse highlights a broader issue with how easily individuals can employ someone else's image for potentially sexual content without consent. As the weeks progressed, the user observed a significant trend: a majority of the most popular videos featuring their likeness were fetish-related. This trend wasn't isolated; similar content could be found using other female creators' images, indicating a pattern in the app's usage that could alienate many users, particularly women. The experience has prompted reflection on the implications of consent in digital spaces. OpenAI, which is still refining Sora, has faced scrutiny over its content moderation practices. Initial concerns focused primarily on misinformation and copyright infringement, yet the emergence of fetish content has raised new ethical dilemmas. While Sora prohibits the creation of videos using the likenesses of living celebrities, it allows depictions of deceased figures, leading to a wave of content featuring well-known personalities from various fields. However, this policy has not been without backlash, as evidenced by a recent pause on content featuring the likeness of Dr. Martin Luther King Jr. following family complaints. The app’s current guidelines also raise questions about the interactions permitted between users, particularly minors and adult content. Recent reports of videos featuring underage individuals alongside adult figures have prompted discussions about the need for stricter controls. OpenAI has announced plans to implement new parental controls for teen accounts, but the specifics of these regulations remain unclear. Ultimately, the controversy surrounding Sora underscores a critical dialogue about the responsibilities of technology companies in managing user-generated content, particularly regarding sensitive themes like sexuality and consent. As the app continues to gain traction, it remains to be seen how OpenAI will navigate these complex challenges.