arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.05273cs.RO
收藏

具有包容性的交互决策与执行以应对模糊指令

Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions

Department of Automation, Tsinghua University(清华大学自动化系);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息学院);Computer Science and Engineering Department, Chinese University of Hong Kong(香港中文大学计算机科学与工程系)

Hengxuan Xu, Fengbo Lan, Zhixin Zhao, Shengjie Wang, Mengqiao Liu, Jieqian Sun, Yu Cheng, Tao Zhang

AI总结:

AIDE通过双流框架整合交互式探索与视觉-语言推理,实现对模糊指令的高效决策与执行,提升机器人在陌生环境中的任务规划能力。

摘要:

使机器人在模糊的人类指令下探索并行动于陌生环境仍是一个挑战,现有基于视觉-语言模型(VLM)的方法在交互式识别任务相关物体(例如,识别杯子或饮料以回应“我口渴”)方面效率低下。这一挑战源于推理效率低下和缺乏环境交互,阻碍了实时任务规划和执行。为此,我们提出了具有包容性的交互决策与执行以应对模糊指令(AIDE),一种双流框架,整合了交互式探索与视觉-语言推理,其中多阶段推理(MSI)作为决策流,加速决策(ADM)作为执行流,能够实现零样本的包容性分析和解释模糊指令。在模拟和现实环境中的广泛实验表明,AIDE在任务规划成功率达到80%以上,并在闭合回路连续执行中以10 Hz频率达到95%以上的准确性,优于现有基于VLM的方法在多样化的开放世界场景中。

英文摘要:

Enabling robots to explore and act in unfamiliar environments under ambiguous human instructions by interactively identifying task-relevant objects (e.g., cups or beverages for "I'm thirsty") remains challenging for existing vision-language model (VLM)-based methods, due to inefficient reasoning and limited environmental interaction. These factors hinder real-time task planning and execution. To address this, we propose Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions (AIDE), a human-inspired dual-stream framework that integrates interactive exploration with vision-language reasoning: mirroring the complementary deliberative and intuitive processes of human cognition, Multi-Stage Inference (MSI) serves as the decision-making stream and Accelerated Decision-Making (ADM) as the execution stream, enabling zero-shot affordance analysis and interpretation of ambiguous instructions. Experiments in simulation and real-world settings show that AIDE achieves over 80% task planning success rate and more than 95% closed-loop continuous execution accuracy at 10 Hz, surpassing existing VLM-based methods in diverse open scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑