TourPhysics：从单张图像将物理引入世界模型以实现探索与操作

中文摘要 AI 辅助

交互式视觉世界模型必须区分观测结果与物理干预：相机运动可揭示新表面，而干预会改变物体运动、接触及形变。当前视频世界模型大多由外观先验驱动，在长时程中常丢失物理或空间一致性。我们提出TourPhysics，这是一个从单张图像和声明式物理配置初始化的在线框架。TourPhysics将PhysOmni（我们在ACM Multimedia 2026上的工作）从有限的基于物理的视频合成扩展为持久的探索与操作框架。TourPhysics将确定性模拟与视频生成相结合，同时为模拟器状态、几何证据、生成器控制及外观记忆分配不同角色：对于每个动作，模拟器会计算有限的物理和相机轨迹，之后生成对应观测结果；被接受的观测结果会发布终端状态，并更新外观记忆及后续生成器控制，而提交的状态与模拟器几何在整个合成与重试过程中保持固定。我们进一步将用于投影和可见性的模拟器几何，与用于条件生成器的相对深度相分离；基于参考的记忆通过几何跨视图对应检索被接受的静态外观，并通过有界残差将其纳入，当不存在有效对应时，该残差会恢复为原生路径。在模拟器定义的相机巡游与物体操作任务上，TourPhysics相比所评估的基线更紧密地遵循规定的相机与物体轨迹，保留输入场景，并在长时程重访期间减少外观漂移。

英文摘要

Interactive visual world models must distinguish observation from physical intervention. Camera motion reveals new surfaces, whereas intervention changes object motion, contact, and deformation. Current video world models are largely driven by appearance priors and often lose physical or spatial consistency over long horizons. We present TourPhysics, an online framework initialized from a single image and a declarative physical configuration. TourPhysics extends PhysOmni, our ACM Multimedia 2026 work, from finite physics-grounded video synthesis to persistent exploration and manipulation. TourPhysics combines deterministic simulation with video generation while assigning separate roles to simulator state, geometric evidence, generator controls, and appearance memory. For each action, the simulator computes a finite physical and camera trajectory before the corresponding observation is generated. Accepted observations publish the terminal state and update the appearance memory and subsequent generator controls, while the committed state and simulator geometry remain fixed throughout synthesis and retry. We further separate the simulator geometry used for projection and visibility from the relative depth used to condition the generator. A reference-anchored memory retrieves accepted static appearance through geometric cross-view correspondence and incorporates it through a bounded residual that reverts to the native path when no valid correspondence exists. On simulator-defined camera tours and object manipulations, TourPhysics follows prescribed camera and object trajectories more closely than the evaluated baselines, preserves the input scene, and reduces appearance drift during long-horizon revisits.

发表机构

• Fudan University（复旦大学）
• Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI））
• Nanjing University（南京大学）

机构由 AI 辅助整理，请以论文原文为准。