Video2Act：一种双系统视频扩散策略，具有机器人空间-运动建模
Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modeling
AI总结：
Video2Act通过整合空间和运动感知表示，提升机器人动作学习效率，其双系统设计在仿真和现实任务中均取得显著成果。
摘要：
鲁棒的感知和动态建模是真实世界机器人策略学习的基础。最近的方法利用视频扩散模型（VDMs）增强机器人策略，提高其对物理世界的理解和建模能力。然而，现有方法忽略了VDMs中跨帧一致且物理一致的运动表示。为此，我们提出了Video2Act框架，通过显式整合空间和运动感知表示，高效引导机器人动作学习。基于VDMs的内在表示，我们提取前景边界和帧间运动变化，同时过滤背景噪声和任务无关的偏差。这些优化后的表示随后作为额外条件输入到扩散变压器（DiT）动作头中，使其能够推理应操纵什么和如何移动。为缓解推理效率问题，我们提出异步双系统设计，其中VDM作为慢速系统2，DiT头作为快速系统1，协同生成适应性动作。通过为系统1提供运动感知条件，Video2Act在VDM低频更新下仍能保持稳定的操控。在评估中，Video2Act在仿真和现实任务中平均成功率分别超越现有最先进VLA方法7.7%和21.7%，进一步表现出强大的泛化能力。
英文摘要：
Robust perception and dynamics modeling are fundamental to real-world robotic policy learning. Recent methods employ video diffusion models (VDMs) to enhance robotic policies, improving their understanding and modeling of the physical world. However, existing approaches overlook the coherent and physically consistent motion representations inherently encoded across frames in VDMs. To this end, we propose Video2Act, a framework that efficiently guides robotic action learning by explicitly integrating spatial and motion-aware representations. Building on the inherent representations of VDMs, we extract foreground boundaries and inter-frame motion variations while filtering out background noise and task-irrelevant biases. These refined representations are then used as additional conditioning inputs to a diffusion transformer (DiT) action head, enabling it to reason about what to manipulate and how to move. To mitigate inference inefficiency, we propose an asynchronous dual-system design, where the VDM functions as the slow System 2 and the DiT head as the fast System 1, working collaboratively to generate adaptive actions. By providing motion-aware conditions to System 1, Video2Act maintains stable manipulation even with low-frequency updates from the VDM. For evaluation, Video2Act surpasses previous state-of-the-art VLA methods by 7.7% in simulation and 21.7% in real-world tasks in terms of average success rate, further exhibiting strong generalization capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
