OpenAI’s Sora 2 lets users insert themselves into AI videos with sound

On Tuesday, OpenAI introduced Sora 2, an advanced video synthesis AI model that marks a significant leap forward in generating videos with synchronized dialogue and sound effects. This innovative feature is a first for the company, enhancing the realism of AI-generated content. Alongside Sora 2, OpenAI launched a new iOS social application that allows users to insert themselves into these AI-generated videos through a feature known as "cameos." During the announcement, OpenAI showcased the capabilities of Sora 2 through a striking video featuring a lifelike representation of CEO Sam Altman, who interacts with the audience against imaginative backdrops, including vibrant scenes like a competitive ride-on duck race. The new model boasts sophisticated capabilities in creating background soundscapes and effects that closely mimic reality. This development follows Google's Veo 3, which was the first of its kind to successfully synchronize audio with video. Shortly before this, Alibaba also released Wan 2.5, an open-weights video model capable of audio generation. With Sora 2, OpenAI now joins the ranks of companies innovating in audio-visual AI technology. Sora 2 also introduces notable enhancements in visual consistency compared to its predecessor. It can now handle more complex instructions over multiple shots while maintaining coherence throughout. OpenAI describes this model as a pivotal moment in video technology, likening its impact to the breakthrough of ChatGPT in text generation. Improvements in physical accuracy are also a highlight of the new model, which can simulate intricate physical movements such as Olympic gymnastics and triple Axels, all while adhering to realistic physics. This update addresses issues seen in earlier versions of OpenAI's video generation models, which often resulted in unrealistic object behavior. OpenAI explained that prior models tended to be overly optimistic, causing objects to behave in ways that defied logic. For instance, if a basketball player missed a shot, previous models might have shown the ball teleporting to the hoop. In contrast, Sora 2 ensures that realistic interactions are preserved, illustrating a basketball rebounding off the backboard if a shot is missed.