arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.09038cs.CVcs.AIcs.GRcs.LG
收藏

生成式视频模型是否理解物理原理?

Do generative video models understand physical principles?

INSAIT Sofia University(INSAIT索菲亚大学);Google DeepMind(谷歌DeepMind)

Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, Robert Geirhos

AI总结:

本研究通过构建Physics-IQ基准数据集,评估多个主流视频生成模型,发现其物理理解能力严重不足且与视觉逼真度无关,表明视觉逼真度不蕴含物理理解。

摘要:

AI视频生成正在经历一场革命,其质量和逼真度迅速提升。这些进展引发了一场激烈的科学辩论:视频模型是否学习了能够发现物理规律的“世界模型”,或者,它们仅仅是复杂的像素预测器,实现了视觉逼真度却并未理解现实的物理原理?我们通过开发Physics-IQ来回答这个问题,这是一个全面的基准数据集,只有获得对各种物理原理(如流体动力学、光学、固体力学、磁学和热力学)的深入理解才能解决。我们发现,在一系列当前模型(Sora、Runway、Pika、Lumiere、Stable Video Diffusion和VideoPoet)中,物理理解严重受限,并且与视觉逼真度无关。与此同时,一些测试用例已经可以被成功解决。这表明,仅通过观察获取某些物理原理可能是可能的,但仍存在重大挑战。虽然我们预计未来会有快速进展,但我们的工作表明,视觉逼真度并不蕴含物理理解。我们的项目页面位于https://physics-iq.github.io;代码位于https://github.com/google-deepmind/physics-IQ-benchmark。

英文摘要:

AI video generation is undergoing a revolution, with quality and realism advancing rapidly. These advances have led to a passionate scientific debate: Do video models learn "world models" that discover laws of physics -- or, alternatively, are they merely sophisticated pixel predictors that achieve visual realism without understanding the physical principles of reality? We address this question by developing Physics-IQ, a comprehensive benchmark dataset that can only be solved by acquiring a deep understanding of various physical principles, like fluid dynamics, optics, solid mechanics, magnetism and thermodynamics. We find that across a range of current models (Sora, Runway, Pika, Lumiere, Stable Video Diffusion, and VideoPoet), physical understanding is severely limited, and unrelated to visual realism. At the same time, some test cases can already be successfully solved. This indicates that acquiring certain physical principles from observation alone may be possible, but significant challenges remain. While we expect rapid advances ahead, our work demonstrates that visual realism does not imply physical understanding. Our project page is at https://physics-iq.github.io; code at https://github.com/google-deepmind/physics-IQ-benchmark.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑