OpenAI's Sora 2 is putting safety and censorship to the test with stunningly real videos

OpenAI's latest video app, Sora 2, has quickly gained popularity, landing at number three on Apple's App Store, following a monumental $6.6 billion share sale that positioned it as the most valuable private company globally. Despite its invite-only release, users are flocking to the TikTok-style platform, which leverages advanced AI technology to create remarkably realistic videos. However, this rapid rise has reignited internal discussions at OpenAI regarding the balance between ensuring safety and fostering creative expression. Sources close to the company's strategy revealed that while leadership believes in maintaining rigorous safety measures, there is a growing concern about potentially stifling innovation or appearing overly censorious. This ongoing tension within the organization remains unresolved. Historically, OpenAI has prioritized speed, often releasing new tools ahead of competitors and allowing real-time public adaptation. A former employee noted that the company's leadership habitually favored quick launches. This strategy was evident when a competitor, China’s DeepSeek, released an efficient AI model late last year, prompting OpenAI to respond rapidly with two new models to maintain its competitive edge. The evolution of OpenAI from a small research lab to a structured entity has enabled it to form cross-functional teams swiftly, enhancing the development and deployment cycles for products like Sora. The app incorporates multiple safety features aimed at preventing harmful content, including prompt filtering and output moderation. It prohibits explicit material, terrorist content, and anything that promotes self-harm, while also using watermarks and disallowing likeness impersonation. Yet, some users have already discovered methods to bypass these safeguards. Sora 2 represents a significant upgrade from its predecessor, producing longer and more coherent clips that closely mimic reality. Viral videos featuring OpenAI CEO Sam Altman have emerged, with his permission granted for the use of his likeness, alongside unsettling portrayals of beloved cartoon characters. This has led to criticism that OpenAI is advancing faster than its protective measures can keep pace. Altman has addressed concerns, stating that Sora is not only about transparency in showcasing technological capabilities but also about generating commercial momentum to support OpenAI’s long-term goals in artificial general intelligence. The launch of Sora comes as competition heats up, with Meta introducing its own short-form AI video feed, Vibes, and Google unveiling Veo 3, alongside rival offerings from ByteDance and Alibaba. OpenAI has also announced a new investment of $850 billion, reinforcing its commitment to infrastructure and next-generation models. Experts argue that the development of video capabilities is crucial for creating more advanced AI systems. Professor Hao Li emphasizes that current AI models primarily learn from linguistic data, but for true general intelligence, they must also assimilate visual and audio information, akin to how infants learn about the world. Research in this area, including the use of AI-generated video to improve model performance, is part of a larger trend among scientists who view video generation as a means to simulate reality and enhance machine reasoning. Former OpenAI executive Zack Kass supports the idea of early deployment of powerful technology, arguing that transparency in technological advancements is essential for collective progress. As the landscape of AI continues to evolve, the conversation surrounding safety, creativity, and the implications of rapid technological advancements remains more pertinent than ever.