arXiv 2512.185712025-12-21cs.AIcs.CV

ESearch-R1: 通过强化学习学习成本感知的多模态大语言模型代理以进行交互式具身搜索

ESearch-R1: Learning Cost-Aware MLLM Agents for Interactive Embodied Search via Reinforcement Learning

School of Traffic and Transportation, Beijing Jiaotong University（交通与运输学院，北京交通大学）；Tencent Robotics X & Futian Laboratory（腾讯机器人X与福田实验室）；Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（基础模型研究中心，中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）

Weijie Zhou, Xuangtang Xiong, Ye Tian, Lijun Yue, Xinyu Wu, Wei Li, Chaoyang Zhao, Honghui Dong, Ming Tang, Jinqiao Wang, Zhengyou Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ESearch-R1通过强化学习方法，结合交互对话、记忆检索和导航，实现成本感知的多模态大语言模型代理，有效降低任务执行成本并提高成功率。

摘要：

多模态大语言模型（MLLMs）已赋予具身代理在规划和推理方面显著的能力。然而，当面对模糊的自然语言指令（例如在杂乱的房间中“获取工具”）时，当前的代理往往无法在物理探索的高成本与人类交互的认知成本之间取得平衡。它们通常将歧义消除视为一个被动感知问题，缺乏减少总任务执行成本的战略推理能力。为此，我们提出了ESearch-R1，一个成本感知的具身推理框架，将交互对话（询问）、片段记忆检索（获取记忆）和物理导航（导航）统一到一个决策过程中。我们引入了HC-GRPO（异构成本感知组相对策略优化）。与传统的PPO不同，HC-GRPO通过采样一组推理轨迹并强化那些在信息增益和异构成本（例如导航时间和人类注意力）之间取得最优权衡的轨迹来优化MLLM。在AI2-THOR中的广泛实验表明，ESearch-R1显著优于标准的ReAct基于代理。它在提高任务成功率的同时，将总操作成本降低了约50%，验证了GRPO在使MLLM代理与物理世界约束对齐方面的有效性。

英文摘要：

Multimodal Large Language Models (MLLMs) have empowered embodied agents with remarkable capabilities in planning and reasoning. However, when facing ambiguous natural language instructions (e.g., "fetch the tool" in a cluttered room), current agents often fail to balance the high cost of physical exploration against the cognitive cost of human interaction. They typically treat disambiguation as a passive perception problem, lacking the strategic reasoning to minimize total task execution costs. To bridge this gap, we propose ESearch-R1, a cost-aware embodied reasoning framework that unifies interactive dialogue (Ask), episodic memory retrieval (GetMemory), and physical navigation (Navigate) into a single decision process. We introduce HC-GRPO (Heterogeneous Cost-Aware Group Relative Policy Optimization). Unlike traditional PPO which relies on a separate value critic, HC-GRPO optimizes the MLLM by sampling groups of reasoning trajectories and reinforcing those that achieve the optimal trade-off between information gain and heterogeneous costs (e.g., navigate time, and human attention). Extensive experiments in AI2-THOR demonstrate that ESearch-R1 significantly outperforms standard ReAct-based agents. It improves task success rates while reducing total operational costs by approximately 50\%, validating the effectiveness of GRPO in aligning MLLM agents with physical world constraints.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。