发表机构
IDEAS Research Institute; Computer Vision Center, Universitat Autònoma de Barcelona; Warsaw University of Technology（IDEAS研究院; 巴塞罗那自治大学计算机视觉中心; 华沙理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文通过探测视频扩散变换器的内部表示，发现物理量可被高精度线性解码且局部存储，并可作转向向量，表明模型在去噪中主动构建物理信息。
AI 中文摘要
视频生成模型能产生惊人的逼真序列，并日益被提议作为世界模型，但最近的基准测试揭示了它们在物理推理方面的显著缺陷。这引发了一个问题：这些模型是内化了物理原理，还是仅仅重现了熟悉的运动模式？我们通过探测视频扩散变换器（DiTs）的内部表示，针对从模拟器导出的、涵盖重力与接触下的运动学和刚体动力学的真实物理量来解决这一问题。我们发现，这些量在去噪过程的早期即可被高精度线性解码，其性能大幅优于直接从模型自身的噪声潜在变量中解码的基线，这表明相关的物理信息是在去噪过程中主动构建的，而非已存在于输入中。此外，我们展示了位于对象上的令牌的激活携带了相关的物理信息，并且定义在多个帧上的量可以从单个潜在帧中读取。因此，信息在令牌序列中被精确地局部化，并且是全局计算但局部存储的。探测还显示出部分外推能力，能够迁移到训练范围之外的场景变化和对象配置，因此它们读取的内容不仅仅是所拟合场景的相关物。当直接在完整分辨率的激活空间中进行拟合时，探测方向可以作为转向向量来改变模型的输出。
英文摘要
Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent benchmarks reveal pronounced deficits in their physical reasoning. This raises the question of whether these models internalize physical principles or merely reproduce familiar motion patterns. We address this by probing internal representations of video Diffusion Transformers (DiTs) for simulator-derived ground-truth physical quantities spanning kinematic motion and rigid-body dynamics under gravity and contact. We find that these quantities are linearly decodable with high accuracy early in the denoising process, substantially outperforming a baseline decoded directly from the model's own noised latents, indicating that the relevant physical information is actively constructed during denoising rather than already present in the input. Additionally, we show that activations at on-object tokens carry the relevant physical information and that quantities defined over multiple frames are readable from single latent frames. Hence, information is sharply localized within the token sequence and is computed globally but stored locally. The probes further show partial extrapolation, transferring to scene variations and object configurations outside their training regime, so what they read is not simply a correlate of the scenes they were fit on. When fitted directly in the full-resolution activation space, the probing directions can serve as steering vectors to change the model's output.
Comments22 pages, 8 figures, 5 tables
