ChatVLA-2：基于预训练知识实现开放世界具身推理的视觉-语言-动作模型
ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge
AI总结：
提出混合专家模型ChatVLA-2及两阶段训练流程，在微调中保留预训练VLM的开放世界推理与空间智能等核心能力，实现可执行推理，在数学匹配等任务上显著超越OpenVLA等现有模仿学习方法。
摘要：
视觉-语言-动作（VLA）模型已成为机器人领域的下一代模型。然而，尽管利用了强大的预训练视觉-语言模型（VLM），现有的端到端 VLA 系统在微调适应特定机器人任务时，往往会丧失关键能力。我们认为，一个可泛化的 VLA 模型应当保留并扩展 VLM 的核心能力：1）开放世界具身推理——VLA 应继承 VLM 的知识，即识别 VLM 能识别的任何事物，具备解决数学问题的能力，并拥有视觉-空间智能；2）推理遵循——有效地将开放世界推理转化为机器人的可执行步骤。在本研究中，我们提出了 ChatVLA-2，这是一种新颖的混合专家 VLA 模型，结合了专用的两阶段训练流程，旨在保留 VLM 原有优势的同时实现可执行推理。为了验证我们的方法，我们设计了一项数学匹配任务，机器人需解读写在白板上的数学问题，并从桌子上挑选相应的数字卡片来解方程。值得注意的是，尽管这些能力未在 VLA 中进行明确训练，我们的方法仍展现出卓越的数学推理和 OCR 能力。此外，我们证明该 VLA 具备强大的空间推理技能，使其能够解读涉及未见物体的新颖方向指令。总体而言，我们的方法展现出的推理与理解能力显著超越了 OpenVLA、DexVLA 和 pi-zero 等最先进的模仿学习方法。这项工作向开发具备稳健推理能力的真正可泛化机器人基础模型迈出了重要一步。
英文摘要：
Vision-language-action (VLA) models have emerged as the next generation of models in robotics. However, despite leveraging powerful pre-trained Vision-Language Models (VLMs), existing end-to-end VLA systems often lose key capabilities during fine-tuning as the model adapts to specific robotic tasks. We argue that a generalizable VLA model should retain and expand upon the VLM's core competencies: 1) Open-world embodied reasoning - the VLA should inherit the knowledge from VLM, i.e., recognize anything that the VLM can recognize, be capable of solving math problems, and possess visual-spatial intelligence, 2) Reasoning following - effectively translating the open-world reasoning into actionable steps for the robot. In this work, we introduce ChatVLA-2, a novel mixture-of-expert VLA model coupled with a specialized two-stage training pipeline designed to preserve the VLM's original strengths while enabling actionable reasoning. To validate our approach, we design a math-matching task wherein a robot interprets math problems written on a whiteboard and picks corresponding number cards from a table to solve equations. Remarkably, our method exhibits exceptional mathematical reasoning and OCR capabilities, despite these abilities not being explicitly trained within the VLA. Furthermore, we demonstrate that the VLA possesses strong spatial reasoning skills, enabling it to interpret novel directional instructions involving previously unseen objects. Overall, our method showcases reasoning and comprehension abilities that significantly surpass state-of-the-art imitation learning methods such as OpenVLA, DexVLA, and pi-zero. This work represents a substantial advancement toward developing truly generalizable robotic foundation models endowed with robust reasoning capacities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
