arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.13383cs.AIcs.LG
收藏

三维迷宫中长期记忆的评价

Evaluating Long-Term Memory in 3D Mazes

Verses Research Lab(Verses研究实验室);DeepMind(深度思维);Google Research(谷歌研究院);University College London(伦敦大学学院);University of Toronto(多伦多大学);University of Berkeley, California(加州大学伯克利分校)

Jurgis Pasukonis, Timothy Lillicrap, Danijar Hafner

摘要:

智能代理需要记住显着的信息,在部分观察的环境中进行推理。例如,具有第一人称视角的智能体应该记住相关对象的位置,即使它们不在视野范围内。类似地,为了有效地在房间中导航,代理需要记住房间如何连接的平面图。然而,强化学习中的大多数基准任务并不测试智能体的长期记忆,这减缓了这一重要研究方向的进展。在本文中,我们介绍了记忆迷宫,一个3D域的随机迷宫专门设计用于评估代理的长期记忆。与现有的基准不同,记忆迷宫测量长期记忆与混淆代理能力分开,并要求代理通过整合信息来定位自己。通过Memory Maze,我们提出了一个在线强化学习基准,一个多样化的离线数据集和一个离线探测评估。记录一个人类玩家建立了一个强大的基线,并验证了建立和保留记忆的必要性,这反映在他们在每一集逐渐增加的奖励中。我们发现,目前的算法受益于训练截断反向传播通过时间和成功的小迷宫,但不符合人类的表现在大型迷宫,留下空间,未来的算法设计进行评估的记忆迷宫。

英文摘要:

Intelligent agents need to remember salient information to reason in partially-observed environments. For example, agents with a first-person view should remember the positions of relevant objects even if they go out of view. Similarly, to effectively navigate through rooms agents need to remember the floor plan of how rooms are connected. However, most benchmark tasks in reinforcement learning do not test long-term memory in agents, slowing down progress in this important research direction. In this paper, we introduce the Memory Maze, a 3D domain of randomized mazes specifically designed for evaluating long-term memory in agents. Unlike existing benchmarks, Memory Maze measures long-term memory separate from confounding agent abilities and requires the agent to localize itself by integrating information over time. With Memory Maze, we propose an online reinforcement learning benchmark, a diverse offline dataset, and an offline probing evaluation. Recording a human player establishes a strong baseline and verifies the need to build up and retain memories, which is reflected in their gradually increasing rewards within each episode. We find that current algorithms benefit from training with truncated backpropagation through time and succeed on small mazes, but fall short of human performance on the large mazes, leaving room for future algorithmic designs to be evaluated on the Memory Maze.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑