KARMA:为具身智能体配备长短期记忆系统
KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systems
AI总结:
针对具身AI智能体执行长序列家务任务时的上下文记忆难题,研究提出整合长短期记忆的KARMA系统,通过记忆增强提示提升LLM规划能力,在AI2-THOR中任务成功率与效率大幅提升,且支持即插即用部署于真实机器人。
摘要:
负责执行相互关联的长序列家务任务的具身AI智能体,往往在上下文记忆方面存在困难,导致任务执行效率低下且错误频发。为解决这一问题,我们提出了KARMA——一种创新的记忆系统,它整合了长期与短期记忆模块,通过记忆增强提示技术强化大语言模型(LLMs)在具身智能体中的规划能力。KARMA对长期记忆和短期记忆进行了区分:长期记忆以全面的3D场景图作为环境的表征,短期记忆则动态记录物体位置与状态的变化。这种双记忆结构使智能体能够检索相关的过往场景经验,从而提升任务规划的准确性与效率。短期记忆采用了高效且自适应的记忆替换策略,在保留关键信息的同时丢弃相关性较低的数据。与当前最先进的搭载记忆增强的具身智能体相比,我们的记忆增强具身AI智能体在AI2-THOR模拟器中的复合任务和复杂任务上,成功率分别提升了1.3倍和2.3倍,任务执行效率分别提升了3.4倍和62.7倍。此外,我们验证了KARMA的即插即用特性,可无缝部署于真实世界机器人系统(如移动操作平台)。凭借这套即插即用的记忆系统,KARMA显著提升了具身智能体生成连贯且符合上下文的规划的能力,让复杂家务任务的执行更高效。本研究的实验视频可在https://youtu.be/4BT7fnw9ehs查看,代码已开源至https://github.com/WZX0Swarm0Robotics/KARMA/tree/master。
英文摘要:
Embodied AI agents responsible for executing interconnected, long-sequence household tasks often face difficulties with in-context memory, leading to inefficiencies and errors in task execution. To address this issue, we introduce KARMA, an innovative memory system that integrates long-term and short-term memory modules, enhancing large language models (LLMs) for planning in embodied agents through memory-augmented prompting. KARMA distinguishes between long-term and short-term memory, with long-term memory capturing comprehensive 3D scene graphs as representations of the environment, while short-term memory dynamically records changes in objects' positions and states. This dual-memory structure allows agents to retrieve relevant past scene experiences, thereby improving the accuracy and efficiency of task planning. Short-term memory employs strategies for effective and adaptive memory replacement, ensuring the retention of critical information while discarding less pertinent data. Compared to state-of-the-art embodied agents enhanced with memory, our memory-augmented embodied AI agent improves success rates by 1.3x and 2.3x in Composite Tasks and Complex Tasks within the AI2-THOR simulator, respectively, and enhances task execution efficiency by 3.4x and 62.7x. Furthermore, we demonstrate that KARMA's plug-and-play capability allows for seamless deployment on real-world robotic systems, such as mobile manipulation platforms.Through this plug-and-play memory system, KARMA significantly enhances the ability of embodied agents to generate coherent and contextually appropriate plans, making the execution of complex household tasks more efficient. The experimental videos from the work can be found at https://youtu.be/4BT7fnw9ehs. Our code is available at https://github.com/WZX0Swarm0Robotics/KARMA/tree/master.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。