Qwen-VLA：统一跨任务、环境和机器人形态的视觉-语言-动作建模
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
- Qwen Team（通义实验室）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出Qwen-VLA，一种基于DiT动作解码器的统一具身基础模型，通过大规模联合预训练和具身感知提示，将操作、导航和轨迹预测统一为动作-轨迹预测框架，实现跨任务、环境和机器人形态的泛化。
AI中文摘要：
具身智能通常通过针对单个任务（如操作或导航）的专用模型进行研究，导致能力碎片化，且跨任务、环境和机器人形态的泛化能力有限。在这项工作中，我们研究了异构的具身决策问题是否可以在单个视觉-语言-动作模型中统一。我们提出了Qwen-VLA，一个统一的具身基础模型，它通过基于DiT的动作解码器将Qwen的视觉-语言建模栈从感知、理解和推理扩展到连续动作和轨迹生成。Qwen-VLA通过大规模联合预训练方案在多样化的数据源上进行训练，包括机器人操作轨迹、人类自我中心演示、合成模拟数据、视觉-语言导航数据、轨迹中心监督和辅助视觉-语言数据。为了支持多种机器人平台，我们引入了具身感知提示调节，其中特定于机器人的文本描述指定了当前的具身形态和控制约定。我们进一步将操作、导航和轨迹预测统一为一个动作-轨迹预测框架，实现了跨机器人形态、任务族和环境的可迁移视觉基础、空间推理和连续动作生成。在操作、导航和轨迹中心基准上的实验显示，在场景布局、背景、光照、物体配置和机器人形态变化下，具有一致的多任务性能和分布外泛化能力。Qwen-VLA-Instruct在LIBERO上达到97.9%，在Simpler-WidowX上达到73.7%，在RoboTwin-Easy/Hard上达到86.1%/87.2%，在R2R上达到69.0% OSR，在RxR上达到59.6% SR，在真实世界ALOHA实验中平均OOD成功率为76.9%，在DOMINO动态操作上零样本成功率为26.6%。
英文摘要：
Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In this work, we study whether heterogeneous embodied decision-making problems can be unified within a single vision-language-action model. We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack from perception, understanding, and reasoning to continuous action and trajectory generation through a DiT-based action decoder. Qwen-VLA is trained with a large-scale joint pretraining recipe over diverse data sources, including robotics manipulation trajectories, human egocentric demonstrations, synthetic simulation data, vision-and-language navigation data, trajectory-centric supervision, and auxiliary vision-language data. To support multiple robot platforms, we introduce embodiment-aware prompt conditioning, where robot-specific textual descriptions specify the current embodiment and control convention. We further cast manipulation, navigation, and trajectory prediction into a unified action-and-trajectory prediction framework, enabling transferable visual grounding, spatial reasoning, and continuous action generation across robot morphologies, task families, and environments. Experiments on manipulation, navigation, and trajectory-centric benchmarks show consistent multi-task performance and out-of-distribution generalization under variations in scene layout, background, lighting, object configuration, and robot embodiment. Qwen-VLA-Instruct achieves 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1%/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA experiments, and 26.6% zero-shot success on DOMINO dynamic manipulation.
