arXiv 2605.136322026-10-02cs.ROcs.CV

引导、思考、行动：面向视觉-语言-动作模型的交互式具身推理

Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models

Futian Laboratory（福田实验室）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；International Digital Economy Academy (IDEA)（国际数字经济学院（IDEA））；School of Robotics, Hunan University（湖南大学机器人学院）；South China University of Technology（华南理工大学）；Visincept（Visincept公司）；National Key Laboratory of Smart Farm Technologies and Systems（智能农业技术与系统国家重点实验室）

Yiran Ling, Qing Lian, Jinghang Li, Qing Jiang, Tianming Zhang, Xiaoke Jiang, Chuanxiu Liu, Jie Liu, Lei Zhang

备注：: Accepted at ECCV 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出GTA-VLA框架，通过允许用户用显式视觉提示引导机器人策略，实现空间可调节的具身推理。该框架整合了外部指导与内部任务规划，提升了在视觉歧义和错误恢复中的表现。

摘要：

在本文中，我们提出了GTA-VLA（引导、思考、行动），一个交互式视觉-语言-动作（VLA）框架，通过允许用户用显式视觉提示引导机器人策略，实现空间可调节的具身推理。现有VLA模型学习一种直接的“感知到行动”映射，从多模态观察到机器人动作。虽然在训练分布内有效，但这种紧密耦合的策略在分布外（OOD）偏移时容易失效，且在发生故障时难以纠正。尽管最近的具身链式思考（CoT）方法暴露了中间推理，但它们仍缺乏将人类空间指导纳入其中的机制，限制了解决视觉歧义或从错误中恢复的能力。为了解决这一差距，我们的框架允许用户通过空间先验（如可及点、框和轨迹）可选地引导策略，后续的推理过程可直接基于这些输入进行条件化。基于这些输入，模型生成统一的空间-视觉链式思考，整合外部指导与内部任务规划，使人类视觉意图与自主决策对齐。对于实际部署，我们进一步将推理模块与轻量级的反应动作头结合，以实现高效的行动执行。广泛的实验表明了我们方法的有效性。在域内SimplerEnv WidowX基准测试中，我们的框架实现了最先进的81.2%的成功率。在分布外视觉偏移和空间歧义下，单次视觉交互显著提高了任务成功率，凸显了交互式推理在具身控制中失败恢复的价值。项目细节可在此处找到：https://signalispupupu.github.io/GTA-VLA_ProjPage/

英文摘要：

In this paper, we propose GTA-VLA(Guide, Think, Act), an interactive Vision-Language-Action (VLA) framework that enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues. Existing VLA models learn a direct "Sense-to-Act" mapping from multimodal observations to robot actions. While effective within the training distribution, such tightly coupled policies are brittle under out-of-domain (OOD) shifts and difficult to correct when failures occur. Although recent embodied Chain-of-Thought (CoT) approaches expose intermediate reasoning, they still lack a mechanism for incorporating human spatial guidance, limiting their ability to resolve visual ambiguities or recover from mistakes. To address this gap, our framework allows users to optionally guide the policy with spatial priors, such as affordance points, boxes, and traces, which the subsequent reasoning process can directly condition on. Based on these inputs, the model generates a unified spatial-visual Chain-of-Thought that integrates external guidance with internal task planning, aligning human visual intent with autonomous decision-making. For practical deployment, we further couple the reasoning module with a lightweight reactive action head for efficient action execution. Extensive experiments demonstrate the effectiveness of our approach. On the in-domain SimplerEnv WidowX benchmark, our framework achieves a state-of-the-art 81.2% success rate. Under OOD visual shifts and spatial ambiguities, a single visual interaction substantially improves task success over existing methods, highlighting the value of interactive reasoning for failure recovery in embodied control. More details of the project can be found here: https://github.com/FutianLabs/GTA-VLA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。