基于长期机器人经验的分层表示的事件记忆语言化
Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experience
AI总结:
本文提出了一种基于分层表示的机器人长期经验语言化方法,利用大规模预训练模型实现零样本学习,通过构建树状数据结构实现事件记忆的自然语言抽象,提升人机交互能力。
摘要:
机器人经验的语言化,即对机器人过去经历的总结和关于机器人过去经历的问题回答,是提高人机交互能力的关键能力。先前的工作应用了基于规则的系统或微调的深度模型来语言化短暂(几分钟长)的事件数据流,限制了泛化和可转移性。在我们的工作中,我们应用了大规模预训练模型来解决这个问题,使用零或少量示例,特别关注于语言化长期经历。为此,我们从事件记忆(EM)中推导出树状数据结构,较低层次表示原始感知和本体感知数据,较高层次抽象事件为自然语言概念。给定这样的由经历流构建的分层表示,我们应用一个大规模语言模型作为代理,根据用户的查询互动搜索EM,动态扩展(最初折叠)的树节点以找到相关信息。该方法在扩展到数月的机器人经验数据时仍能保持计算成本低。我们在模拟家庭机器人数据、人类自体视觉视频和真实世界机器人记录上评估了我们的方法,证明了其灵活性和可扩展性。
英文摘要:
Verbalization of robot experience, i.e., summarization of and question answering about a robot's past, is a crucial ability for improving human-robot interaction. Previous works applied rule-based systems or fine-tuned deep models to verbalize short (several-minute-long) streams of episodic data, limiting generalization and transferability. In our work, we apply large pretrained models to tackle this task with zero or few examples, and specifically focus on verbalizing life-long experiences. For this, we derive a tree-like data structure from episodic memory (EM), with lower levels representing raw perception and proprioception data, and higher levels abstracting events to natural language concepts. Given such a hierarchical representation built from the experience stream, we apply a large language model as an agent to interactively search the EM given a user's query, dynamically expanding (initially collapsed) tree nodes to find the relevant information. The approach keeps computational costs low even when scaling to months of robot experience data. We evaluate our method on simulated household robot data, human egocentric videos, and real-world robot recordings, demonstrating its flexibility and scalability.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。