ViPSim: 协同视觉与参数空间实现一致的长时程具身世界模型
ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models
AI总结:
提出ViPSim框架,通过协同视觉空间(显式空间先验)和参数空间(数值驱动)解决长时程视频生成中的轨迹漂移和不一致问题,实现高保真具身世界模型。
摘要:
具身世界模型(EWMs)已成为一种可扩展且无风险的范式,用于推进具身智能,支持对视觉-语言-动作系统的安全性关键评估。然而,它们作为评估基准和基础模拟器的可靠性常常受到低维动作与高维视频合成之间表示差距的阻碍。这种差距导致缺乏几何对应性,表现为长时程展开中的累积轨迹漂移和不一致的机器人-物体交互。为弥合这一差距,我们提出ViPSim,一个通过视觉空间和参数空间的协同协作实现一致长时程生成的框架。我们将视觉空间定义为显式空间先验的领域,整合了末端执行器姿态的像素对齐投影、相机视角、深度感知的场景几何以及机器人形态掩码,以提供密集的结构化基础。同时,参数空间作为数值驱动领域,注入原始动作序列和相机矩阵,以提供精确的运动引导。通过统一这两个空间,ViPSim确保生成的状态同时由几何边界锚定并由数值命令引导。大量实验表明,ViPSim与骨干网络无关,并显著增强了轨迹一致性。值得注意的是,我们的方法在生成与可变形物体(如布料折叠)的复杂交互方面展现出涌现能力,并在分布外和跨具身场景中保持稳健性能,为具身智能体的自动化评估和预测控制提供了高保真基础。
英文摘要:
Embodied World Models (EWMs) have emerged as a scalable and risk-free paradigm for advancing embodied intelligence, enabling the safety-critical evaluation of Vision-Language-Action systems. However, their reliability as evaluation benchmarks and foundational simulators is often hindered by the representation gap between low-dimensional actions and high-dimensional video synthesis. This gap results in a lack of geometric correspondence, manifesting as accumulated trajectory drift and inconsistent robot-object interactions during long-horizon rollouts. To bridge this gap, we propose ViPSim, a framework that achieves consistent long-horizon generation through the synergistic collaboration of Visual and Parameter Spaces. We define the Visual Space as a domain of explicit spatial priors, integrating pixel-aligned projections of end-effector pose, camera perspectives, depth-informed scene geometry, and robotic morphological masks to provide dense structural grounding. Concurrently, the Parameter Space serves as a domain of numerical drivers, injecting raw action sequences and camera matrices to provide precise motion guidance. By unifying these two spaces, ViPSim ensures that the generated states are simultaneously anchored by geometric boundaries and steered by numerical commands. Extensive experiments demonstrate that ViPSim is backbone-agnostic and significantly enhances trajectory consistency. Notably, our approach exhibits emergent capabilities in generating complex interactions with deformable objects (e.g., cloth folding) and maintains robust performance in out-of-distribution and cross-embodiment scenarios, providing a high-fidelity foundation for the automated evaluation and predictive control of embodied agents.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。