探索性检索增强规划用于持续具身指令跟随
Exploratory Retrieval-Augmented Planning For Continual Embodied Instruction Following
AI总结：
本文提出ExRAP框架，通过环境探索与记忆增强查询评估，结合时间一致性细化，提升LLM在动态环境中持续指令跟随的规划能力，在多个基准上优于现有方法。
摘要：
本研究提出了一种探索性检索增强规划（ExRAP）框架，旨在解决动态、非平稳环境中具身智能体的持续指令跟随任务。该框架通过高效探索物理环境并建立环境上下文记忆，增强了大语言模型（LLM）的具身推理能力，从而有效地将任务规划过程锚定在时变的环境上下文中。在ExRAP中，给定多个持续指令跟随任务，每个指令被分解为对环境上下文记忆的查询和基于查询结果的任务执行。为了高效处理这些持续且同时进行的多个任务，我们通过将基于信息的探索（information-based exploration）纳入基于LLM的规划过程，实现了一种探索集成的任务规划方案。结合记忆增强的查询评估，这种集成方案不仅能够更好地平衡环境上下文记忆的有效性与环境探索的负载，而且提高了整体任务性能。此外，我们设计了一种时间一致性细化（temporal consistency refinement）方案用于查询评估，以应对记忆中固有的知识衰减。通过在VirtualHome、ALFRED和CARLA上的实验，我们的方法在涉及不同指令规模和类型以及不同非平稳程度的多种具身指令跟随场景中表现出鲁棒性，并且在目标成功率和执行效率方面始终优于其他最先进的基于LLM的任务规划方法。
英文摘要：
This study presents an Exploratory Retrieval-Augmented Planning (ExRAP) framework, designed to tackle continual instruction following tasks of embodied agents in dynamic, non-stationary environments. The framework enhances Large Language Models' (LLMs) embodied reasoning capabilities by efficiently exploring the physical environment and establishing the environmental context memory, thereby effectively grounding the task planning process in time-varying environment contexts. In ExRAP, given multiple continual instruction following tasks, each instruction is decomposed into queries on the environmental context memory and task executions conditioned on the query results. To efficiently handle these multiple tasks that are performed continuously and simultaneously, we implement an exploration-integrated task planning scheme by incorporating the {information-based exploration} into the LLM-based planning process. Combined with memory-augmented query evaluation, this integrated scheme not only allows for a better balance between the validity of the environmental context memory and the load of environment exploration, but also improves overall task performance. Furthermore, we devise a {temporal consistency refinement} scheme for query evaluation to address the inherent decay of knowledge in the memory. Through experiments with VirtualHome, ALFRED, and CARLA, our approach demonstrates robustness against a variety of embodied instruction following scenarios involving different instruction scales and types, and non-stationarity degrees, and it consistently outperforms other state-of-the-art LLM-based task planning approaches in terms of both goal success rate and execution efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
