Decart’s new world model can simulate hours of photorealistic driving — with some caveats

On Wednesday, Decart, an AI startup, introduced Oasis 3, its newest interactive world model designed to create photorealistic driving environments in real-time. This innovative model is currently accessible via an API, primarily aimed at autonomous vehicle companies needing to replicate rare driving scenarios on a large scale. The startup has plans to broaden its applications into robotics and various physical AI sectors. Decart is making a significant move by providing API access from the outset, aiming to cultivate a developer ecosystem akin to the one OpenAI established with its language models. Dean Leitersdorf, co-founder and CEO of Decart, expressed enthusiasm about this venture, stating, “It’s going to be the first usable world model that people can actually program on top of.” The company already boasts a community of over 100,000 developers, many of whom are utilizing its previous real-time video model, Lucy, for applications in e-commerce and live streaming. Oasis 3 builds upon this foundation and signifies Decart's ambitious entry into the physical AI domain. The access pricing is set at $0.02 per second, with enterprise rates varying based on specific use cases. The startup operates in a competitive landscape, with Google’s Genie 3 and other emerging video generation models striving for prominence in the world model arena. The release of Oasis 3 follows Decart's successful funding round, which raised $300 million and increased the company's valuation to nearly $4 billion. Notable investors include Toyota, Adobe, and eBay, all of whom represent potential customers. The efficiency of Oasis 3 stems from Decart's proprietary DOS (Decart Optimization Stack) software, which enables the models to run seamlessly on platforms like Nvidia, Amazon, and Google, significantly reducing operational costs compared to competitors. Oasis 3 offers physically accurate environments with multiple camera perspectives for autonomous vehicle training and testing. Unlike limited demos from other models, Decart allows developers to generate an infinite number of scenarios, an essential feature for those working on autonomous vehicle applications. Preliminary tests indicate that Oasis 3 delivers superior photorealism, particularly when responding to a single text prompt. However, as users interact with the model over extended periods, the experience can deteriorate. For instance, while a realistic New York City street can be generated initially, the surrounding environment may lose its thematic coherence during navigation. The model struggles with consistent physics simulation, with vehicles often passing through one another, a challenge that Leitersdorf acknowledges as a significant research hurdle to overcome. The architecture of Oasis 3 is auto-regressive, generating one frame at a time and referencing previous frames to create continuity. This method is inherently resource-intensive, leading to rapid context window saturation. To address this, the Decart team is researching ways to enhance the model's memory capabilities. Looking ahead, Leitersdorf is optimistic about future versions of Oasis 3, which may utilize video inputs for environment generation, potentially resolving some current limitations. He sees parallels to the early days of large language models, believing that as developers experiment with Oasis 3, they will uncover innovative applications that could surprise the industry.