Luma releases a new AI model that lets users generate a video from a start and end frame

Luma, a company backed by a16z that specializes in AI-driven video and 3D modeling, has launched an innovative model named Ray3 Modify. This new tool empowers users to enhance existing footage by simply supplying character reference images, ensuring that the essence of the original performance remains intact. The Ray3 Modify model allows creators to specify a starting and ending frame to guide the generation of transitional footage. This capability addresses a common challenge in the industry: preserving the nuances of human performance while utilizing AI for creative editing. According to Luma, the model closely adheres to the original input, making it ideal for studios that wish to incorporate human actors into their creative projects. Amit Jain, co-founder and CEO of Luma AI, expressed excitement about the model's potential. "Generative video models are incredibly expressive but also hard to control. Today, we are thrilled to introduce Ray3 Modify, which blends the real-world with the expressivity of AI while granting full control to creatives. This means that creative teams can capture performances and then modify them to fit any setting, change costumes, or even reshoot scenes using AI, all without needing a physical reshoot," Jain stated. The model retains critical elements such as the actor’s motion, timing, eye line, and emotional delivery, all while transforming the scene to align with the creator's vision. Additionally, the ability to provide a character reference facilitates the preservation of attributes like costumes and identities throughout the shoot. Ray3 Modify is now accessible via Luma's Dream Machine platform. This release follows a significant funding round in November amounting to $900 million, spearheaded by Humain, an AI firm owned by Saudi Arabia’s Public Investment Fund. Notable investors, including a16z, Amplify Partners, and Matrix Partners, also contributed to this funding effort. Furthermore, Luma is looking to establish a 2GW AI cluster in Saudi Arabia in collaboration with Humain.