Can today’s AI video models accurately model how the real world works?

In recent months, the spotlight has shifted towards the fascinating realm of generative video models, with enthusiasts eager to explore their potential to grasp the physical properties of our world. This capability could lay the groundwork for a sophisticated 'world model,' marking a significant advancement in the real-world applications of generative AI. Google's DeepMind Research has stepped up to examine how effectively these video models can learn from training data about the real world. In their recent publication, titled "Video Models are Zero-shot Learners and Reasoners," the team employed Google's Veo 3 model to produce a multitude of videos aimed at assessing its performance across various tasks related to perception, modeling, manipulation, and reasoning about reality. The researchers confidently assert that Veo 3 is capable of tackling a wide range of challenges without specific training—hence the term "zero-shot." They suggest that these video models are progressing towards becoming comprehensive, generalist vision foundation models. However, a closer inspection of their experimental results indicates that they may be evaluating today's video models with some leniency, anticipating that future advancements will address many of the inconsistencies observed. Notably, Veo 3 does deliver impressive and consistent outcomes in several tasks. For instance, the model reliably generated realistic videos of actions, such as robotic hands successfully opening a jar or catching a ball, across 12 separate trials. Additionally, Veo 3 demonstrated exceptional performance in tasks such as image deblurring, denoising, filling gaps in complex visuals, and edge detection, achieving near-perfect results in these areas.