ELMUR：带有更新/重写的外部层记忆用于长周期强化学习问题
ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problems
AI总结：
ELMUR通过结构化外部记忆提升长周期强化学习任务中的决策能力，实现显著性能提升。
摘要：
现实中的机器人代理必须在部分可观测性和长周期下行动，其中关键线索可能在影响决策之前很久就会出现。然而，大多数现代方法仅依赖于即时信息，而不整合过去的信息。标准的循环或变换器模型在保留和利用长期依赖性方面存在困难：上下文窗口截断历史，而朴素的记忆扩展在规模和稀疏性下失效。我们提出了ELMUR（带有更新/重写的外部层记忆），一种带有结构化外部记忆的变换器架构。每个层维护内存嵌入，通过双向交叉注意与它们交互，并通过使用替换或凸融合的最近最少使用（LRU）内存模块更新它们。ELMUR将有效周期扩展到注意力窗口的10万倍以上，并在具有长达一百万步走廊的合成T-Maze任务中实现了100%的成功率。在POPGym中，它在超过一半的任务上优于基线。在MIKASA-Robo稀疏奖励操作任务中，使用视觉观测时，它几乎将强基线的性能翻倍，在21个任务中实现了最佳成功率，并将所有任务的总体成功率提高了约70%。这些结果表明，结构化的、层本地的外部记忆为部分可观测性下的决策提供了一种简单且可扩展的方法。代码和项目页面：https://elmur-paper.github.io/。
英文摘要：
Real-world robotic agents must act under partial observability and long horizons, where key cues may appear long before they affect decision making. However, most modern approaches rely solely on instantaneous information, without incorporating insights from the past. Standard recurrent or transformer models struggle with retaining and leveraging long-term dependencies: context windows truncate history, while naive memory extensions fail under scale and sparsity. We propose ELMUR (External Layer Memory with Update/Rewrite), a transformer architecture with structured external memory. Each layer maintains memory embeddings, interacts with them via bidirectional cross-attention, and updates them through an Least Recently Used (LRU) memory module using replacement or convex blending. ELMUR extends effective horizons up to 100,000 times beyond the attention window and achieves a 100% success rate on a synthetic T-Maze task with corridors up to one million steps. In POPGym, it outperforms baselines on more than half of the tasks. On MIKASA-Robo sparse-reward manipulation tasks with visual observations, it nearly doubles the performance of strong baselines, achieving the best success rate on 21 out of 23 tasks and improving the aggregate success rate across all tasks by about 70% over the previous best baseline. These results demonstrate that structured, layer-local external memory offers a simple and scalable approach to decision making under partial observability. Code and project page: https://elmur-paper.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
