I built marshmallow castles in Google’s new AI world generator

Google DeepMind has launched Project Genie, an innovative AI tool that allows users to create interactive gaming environments using text prompts or images. This experimental prototype is available to Google AI Ultra subscribers in the U.S., starting Thursday. It harnesses the capabilities of Google’s advanced world model, Genie 3, alongside its image generation model, Nano Banana Pro, and the Gemini framework. This rollout occurs five months after the initial research preview of Genie 3 and is part of DeepMind's strategy to collect user feedback and training data. The development of world models, which are essential for predicting outcomes and planning actions, is viewed as a significant milestone on the path to achieving artificial general intelligence (AGI). In the immediate future, DeepMind aims to leverage these models in video games and entertainment while also considering their applications in training robotic agents within simulated environments. The race to enhance world models is intensifying, with other players in the field, such as Fei-Fei Li's World Labs and Runway’s AI video generation startup, also making strides in this area. Yann LeCun’s AMI Labs is expected to focus on similar advancements. Shlomi Fruchter, a research director at DeepMind, expressed enthusiasm about making the tool accessible to a broader audience and gathering valuable feedback. While Project Genie showcases impressive capabilities, it is still in the experimental stage, meaning users may encounter inconsistencies. For instance, the AI may successfully generate playable worlds at times, but it can also produce unexpected and confusing results. Users begin by sketching their desired world through text prompts that describe the environment and a main character, which they can later control from a first or third-person perspective. The Nano Banana Pro model creates an image based on these prompts, serving as a foundation for building an interactive world, though not all modifications go as planned. Once users finalize their image, Project Genie generates an explorable world within seconds. They can also remix existing worlds or explore curated options for inspiration. Currently, users are limited to 60 seconds of navigation, a constraint imposed by budget and compute limitations. According to Fruchter, this restriction aims to enhance user access while maintaining the quality of the experience. Safety measures are in place, preventing the generation of inappropriate content or characters that infringe on copyrights, a concern raised previously by Disney regarding AI-generated outputs. Despite these limitations, the demo highlights the potential of Project Genie, as users can build whimsical worlds based on their childhood fantasies. However, challenges remain, especially with creating photorealistic environments, which often resemble video game graphics rather than lifelike settings. Users have reported mixed results when instructing the AI with real-life photos, indicating areas for improvement in how the model interprets and generates based on actual images. Interactivity is also a work in progress, with users occasionally experiencing characters passing through solid objects. As DeepMind continues to refine Project Genie, Fruchter remains optimistic about the future enhancements that could improve realism and user interaction. He emphasized that while this tool is not yet a fully realized product, it offers a unique glimpse into a promising future for AI in gaming and creative exploration.