arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07050cs.CVcs.AI
收藏

视频世界模型中的物理解释

Interpreting Physics in Video World Models

FAIR, Meta Superintelligence Labs;Mila \& McGill University;Kempner Institute, Harvard University

Sonia Joseph, Quentin Garrido, Randall Balestriero, Matthew Kowal, Thomas Fel, Shahab Bakhtiari, Blake Richards, Mike Rabbat

AI总结:

该研究通过分析视频模型内部的物理表示,发现其采用分布式表示而非分解表示,揭示了物理信息在模型中的涌现机制和组织方式。

摘要:

在物理推理中,一个长期存在的问题是视频基于模型是否需要依赖物理变量的分解表示才能做出准确的物理预测,或者是否可以隐式地以任务特定的分布式方式表示这些变量。尽管现代视频世界模型在直观物理基准测试中表现强劲,但尚不清楚这些模型内部实现了哪种表示范式。在这里,我们首次进行了解释性研究,直接检查大规模视频编码器中的物理表示。通过逐层探测、子空间几何、块级解码和目标注意力消融,我们确定了物理信息何时变得可访问以及如何在编码器基于视频转换器中组织。在不同架构中,我们识别出一个明显的中间深度过渡——我们称之为物理涌现区——在该区域物理变量变得可访问。与物理相关的表示在该过渡后不久达到峰值,然后在输出层退化。将运动分解为显式变量,我们发现速度和加速度等标量量在早期层即可获得,而运动方向仅在物理涌现区变得可访问。值得注意的是,我们发现方向是通过高维群体结构以圆形几何方式编码的,需要协调多特征干预才能控制。这些发现表明,现代视频模型不像经典物理引擎那样使用物理变量的分解表示。相反,它们使用的是分布式表示,尽管如此,这种表示仍然足以做出物理预测。

英文摘要:

A long-standing question in physical reasoning is whether video-based models need to rely on factorized representations of physical variables in order to make physically accurate predictions, or whether they can implicitly represent such variables in a task-specific, distributed manner. While modern video world models achieve strong performance on intuitive physics benchmarks, it remains unclear which of these representational regimes they implement internally. Here, we present the first interpretability study to directly examine physical representations inside large-scale video encoders. Using layerwise probing, subspace geometry, patch-level decoding, and targeted attention ablations, we characterize where physical information becomes accessible and how it is organized within encoder-based video transformers. Across architectures, we identify a sharp intermediate-depth transition -- which we call the Physics Emergence Zone -- at which physical variables become accessible. Physics-related representations peak shortly after this transition and degrade toward the output layers. Decomposing motion into explicit variables, we find that scalar quantities such as speed and acceleration are available from early layers onwards, whereas motion direction becomes accessible only at the Physics Emergence Zone. Notably, we find that direction is encoded through a high-dimensional population structure with circular geometry, requiring coordinated multi-feature intervention to control. These findings suggest that modern video models do not use factorized representations of physical variables like a classical physics engine. Instead, they use a distributed representation that is nonetheless sufficient for making physical predictions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑