arXiv 2512.180072025-12-19cs.ROcs.CV

机器人VLA通过与运动图像扩散的联合学习获益

Robotic VLA Benefits from Joint Learning with Motion Image Diffusion

Salesforce AI Research（Salesforce人工智能研究）

Yu Fang, Kanchana Ranasinghe, Le Xue, Honglu Zhou, Juntao Tan, Ran Xu, Shelby Heinecke, Caiming Xiong, Silvio Savarese, Daniel Szafir, Mingyu Ding, Michael S. R… 展开作者

Yu Fang, Kanchana Ranasinghe, Le Xue, Honglu Zhou, Juntao Tan, Ran Xu, Shelby Heinecke, Caiming Xiong, Silvio Savarese, Daniel Szafir, Mingyu Ding, Michael S. Ryoo, Juan Carlos Niebles

备注：: Website: https://vla-motion.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出通过联合学习与运动图像扩散提升机器人VLA模型的运动推理能力，实验表明在多个基准测试中显著提升了性能。

摘要：

视觉-语言-动作（VLA）模型通过将多模态观察和指令直接映射到动作，在机器人操作中取得了显著进展。然而，它们通常模仿专家轨迹而没有预测运动推理，这限制了它们对应该采取哪些动作的推理能力。为了解决这一限制，我们提出了联合学习与运动图像扩散，一种新的策略，通过增强VLA模型的运动推理能力。我们的方法在VLA架构中引入了双头设计：虽然动作头预测动作块，如传统的VLA，一个额外的运动头，以扩散变换器（DiT）实现，预测基于光学流的运动图像，以捕捉未来的动态。两个头共同训练，使共享的VLM主干学习将机器人控制与运动知识相结合的表示。这种联合学习在不修改标准VLA推理路径的情况下，构建了时间一致且物理合理的表示。在模拟和现实环境中进行的实验表明，联合学习与运动图像扩散将pi系列VLA的成功率提高到LIBERO基准的97.5%和RoboTwin基准的58.0%，在现实世界性能上提高了23%，验证了其在增强大规模VLA运动推理能力方面的有效性。

英文摘要：

Vision-Language-Action (VLA) models have achieved remarkable progress in robotic manipulation by mapping multimodal observations and instructions directly to actions. However, they typically mimic expert trajectories without predictive motion reasoning, which limits their ability to reason about what actions to take. To address this limitation, we propose joint learning with motion image diffusion, a novel strategy that enhances VLA models with motion reasoning capabilities. Our method extends the VLA architecture with a dual-head design: while the action head predicts action chunks as in vanilla VLAs, an additional motion head, implemented as a Diffusion Transformer (DiT), predicts optical-flow-based motion images that capture future dynamics. The two heads are trained jointly, enabling the shared VLM backbone to learn representations that couple robot control with motion knowledge. This joint learning builds temporally coherent and physically grounded representations without modifying the inference pathway of standard VLAs, thereby maintaining test-time latency. Experiments in both simulation and real-world environments demonstrate that joint learning with motion image diffusion improves the success rate of pi-series VLAs to 97.5% on the LIBERO benchmark and 58.0% on the RoboTwin benchmark, yielding a 23% improvement in real-world performance and validating its effectiveness in enhancing the motion reasoning capability of large-scale VLAs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文