AI 中文总结
本文提出ATLAS算法，通过结构解耦转移动态与奖励信号，在空间导航任务中实现高样本效率、抗灾难性遗忘及对新目标的快速适应，在非平稳环境中性能显著优于基线强化学习算法。
AI 中文摘要
当代无模型强化学习算法可实现极高性能，但样本效率低且对环境变化鲁棒性差；基于模型的算法样本效率高，但环境变化时仍会失效。本文提出基于抽象后继的自适应拓扑学习（ATLAS）以应对这些挑战。ATLAS采用按需生长网络与后继特征，以实现高样本效率，同时稳健应对灾难性遗忘。我们在空间导航任务中评估ATLAS，将其性能与常见的在线策略和离线策略算法进行基准测试。实证结果表明，通过将转移动态与奖励信号在结构上解耦，ATLAS可实现对新目标的近乎即时适应，并能表现出正向反向迁移，在非平稳环境中显著优于基线方法。
英文摘要
Contemporary model-free reinforcement learning algorithms can achieve very high performance, but have low sample efficiency and are not robust to changes in the environment. Model-based algorithms have much higher sample efficiency, but still fail when the environment shifts. This paper introduces Adaptive Topological Learning with Abstract Successors (ATLAS) to combat these challenges. ATLAS uses a Grow When Required network with Successor Features in order to achieve high sample efficiency while also robustly tackling catastrophic forgetting. We evaluate ATLAS in spatial navigation tasks, benchmarking its performance against common on-policy and off-policy algorithms. Our empirical results demonstrate that by structurally decoupling transition dynamics from the reward signal, ATLAS achieves near-instantaneous adaptation to new goals and can exhibit positive backward transfer, significantly outperforming baseline methods in non-stationary environments.
