arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03256cs.CLcs.AI
收藏

E2CL:面向具身智能体的基于探索的纠错学习

E2CL: Exploration-based Error Correction Learning for Embodied Agents

The Hong Kong Polytechnic University(香港理工大学)

Hanlin Wang, Chak Tou Leong, Jian Wang, Wenjie Li

AI总结:

针对语言模型作为具身智能体时内在知识与环境知识错位的问题,提出E2CL框架,通过探索引发的错误与环境反馈实现自我纠错,在VirtualHome中验证了其优于基线方法的环境对齐和自我纠正能力。

摘要:

语言模型在知识利用和推理方面展现出越来越强的能力。然而,当它们作为智能体应用于具身环境时,其内在知识与环境知识之间常常存在错位,从而导致不可行的动作。传统的环境对齐方法,例如基于专家轨迹的监督学习和强化学习,分别在覆盖环境知识和实现高效收敛方面存在局限。受人类学习的启发,我们提出了基于探索的纠错学习(E2CL),这是一个新颖的框架,利用探索引发的错误和环境反馈来增强具身智能体的环境对齐能力。E2CL结合了教师引导的探索和无教师引导的探索,以收集环境反馈并纠正错误动作。智能体学习提供反馈并进行自我纠正,从而增强其对目标环境的适应性。在VirtualHome环境中进行的大量实验表明,经过E2CL训练的智能体优于基线方法训练的智能体,并展现出更优越的自我纠正能力。

英文摘要:

Language models are exhibiting increasing capability in knowledge utilization and reasoning. However, when applied as agents in embodied environments, they often suffer from misalignment between their intrinsic knowledge and environmental knowledge, leading to infeasible actions. Traditional environment alignment methods, such as supervised learning on expert trajectories and reinforcement learning, encounter limitations in covering environmental knowledge and achieving efficient convergence, respectively. Inspired by human learning, we propose Exploration-based Error Correction Learning (E2CL), a novel framework that leverages exploration-induced errors and environmental feedback to enhance environment alignment for embodied agents. E2CL incorporates teacher-guided and teacher-free explorations to gather environmental feedback and correct erroneous actions. The agent learns to provide feedback and self-correct, thereby enhancing its adaptability to target environments. Extensive experiments in the VirtualHome environment demonstrate that E2CL-trained agents outperform those trained by baseline methods and exhibit superior self-correction capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑