arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02456cs.RO
收藏

InternVLA-A1:统一理解、生成和行动以实现机器人操作

InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation

InternVLA-A1 Team(InternVLA-A1团队)
Junhao Cai, Zetao Cai, Jiafei Cao, Yilun Chen, Zeyu He, Lei Jiang, Hang Li, Hengjie Li, Yang Li, Yufei Liu, Yanan Lu, Qi Lv, Haoxiang Ma, Jiangmiao Pang, Yu Qia… 展开作者

Junhao Cai, Zetao Cai, Jiafei Cao, Yilun Chen, Zeyu He, Lei Jiang, Hang Li, Hengjie Li, Yang Li, Yufei Liu, Yanan Lu, Qi Lv, Haoxiang Ma, Jiangmiao Pang, Yu Qiao, Zherui Qiu, Yanqing Shen, Xu Shi, Yang Tian, Bolun Wang, Hanqing Wang, Jiaheng Wang, Tai Wang, Xueyuan Wei, Chao Wu, Yiman Xie, Boyang Xing, Yuqiang Yang, Yuyin Yang, Qiaojun Yu, Feng Yuan, Jia Zeng, Jingjing Zhang, Shenghan Zhang, Shi Zhang, Zhuoma Zhaxi, Bowen Zhou, Yuanzhen Zhou, Yunsong Zhou, Hongrui Zhu, Yangkun Zhu, Yuchen Zhu

AI总结:

InternVLA-A1通过统一的Transformer架构,结合语义理解和动态预测能力,提升了机器人操作任务的性能。

摘要:

现有的视觉-语言-行动(VLA)模型通常基于多模态大语言模型(MLLMs),在语义理解方面表现出色,但本质上缺乏推断物理世界动态的能力。因此,近期的方法转向了世界模型,通常通过视频预测来构建;然而,这些方法往往在语义上缺乏依据,并且在视频预测错误的情况下显得脆弱。为了将语义理解与动态预测能力相结合,我们提出了InternVLA-A1。该模型采用统一的Transformer混合架构,协调三个专家进行场景理解、视觉前瞻生成和行动执行。这些组件通过统一的掩码自注意力机制无缝交互。基于InternVL3和Qwen3-VL,我们将在2B和3B参数规模上实例化InternVLA-A1。我们在这三种数据源上预训练这些模型:真实世界机器人数据、合成模拟数据和人类视频,涵盖超过692M帧。这种混合训练策略有效地利用了合成模拟数据的多样性,同时最小化了仿真到现实的差距。我们在12个真实世界机器人任务和一个仿真基准上评估了InternVLA-A1。结果表明,InternVLA-A1在静态操作任务上比pi0.5高出4.4%,在RoboTwin 2.0仿真基准上高出2.6%,并在动态操作任务上实现了26.7%的提升。

英文摘要:

Prevalent Vision-Language-Action (VLA) models are typically built upon Multimodal Large Language Models (MLLMs) and demonstrate exceptional proficiency in semantic understanding, but they inherently lack the capability to deduce physical world dynamics. Consequently, recent approaches have shifted toward World Models, typically formulated via video prediction; however, these methods often suffer from a lack of semantic grounding and exhibit brittleness in the presence of video prediction errors. To synergize semantic understanding with dynamic predictive capabilities, we present InternVLA-A1. This model employs a unified Mixture-of-Transformers architecture, coordinating three experts for scene understanding, visual foresight generation, and action execution. These components interact seamlessly through a unified masked self attention mechanism. Building upon InternVL3 and Qwen3-VL, we instantiate InternVLA-A1 at 2B and 3B parameter scales. We pre-train these models on heterogeneous data sources over real-world robot data, synthetic simulation data, and human videos, covering over 692M frames. This hybrid training strategy effectively harnesses the diversity of synthetic simulation data while minimizing the sim-to-real gap. We evaluated InternVLA-A1 on 12 real-world robotic tasks and a simulation benchmark. The results show that InternVLA-A1 consistently outperforms prior leading models: compared with pi0.5, it achieves +4.4\% on static manipulation tasks and +2.6\% on the RoboTwin 2.0 simulation benchmark, and delivers a +26.7\% boost on dynamic manipulation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑