发表机构
University of Alberta; Alberta Machine Intelligence Institute (Amii)（阿尔伯塔大学; 阿尔伯塔机器智能研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出Wayfarer，一种通过拉普拉斯表示学习发现选项的通用深度RL智能体，在Atari 2600游戏中显著提升探索、信用分配和泛化，实现最先进性能。
AI 中文摘要
时间抽象，通常以选项的形式实现，长期以来被视为一种加速强化学习（RL）中信用分配、促进探索和实现泛化的机制。然而，开发在大规模、高维领域中有效的一般性选项发现方法仍然是一个根本性挑战。现有的选项发现方法要么局限于相对简单的领域，依赖于手工制作或准符号表示，要么相对于无选项学习几乎没有改进。我们提出了Wayfarer，一个通用的、领域无关的、在线深度强化学习智能体，它通过从高维观测中进行拉普拉斯表示学习来发现选项，并利用这些选项进行控制。我们表明，由此产生的选项同时改善了探索，加速了信用分配，并有效地泛化到未见过的设置，从而能够显著更快地学习复杂策略。Wayfarer在最具挑战性的Atari 2600游戏上实现了单流智能体中的最先进性能，在需要长视野探索和战略行为的游戏（如Montezuma's Revenge和Private Eye）中获得了最大的提升。
英文摘要
Temporal abstractions, often instantiated as options, have long been regarded as a mechanism for accelerating credit assignment, facilitating exploration, and enabling generalisation in reinforcement learning (RL). However, developing general option discovery methods that are effective in large-scale, high-dimensional domains remains a fundamental challenge. Existing option discovery methods are either confined to relatively simple domains, depend on handcrafted or quasi-symbolic representations, or offer little improvement over learning without options. We present Wayfarer, a general, domain-agnostic, online deep RL agent that discovers options through Laplacian representation learning from high-dimensional observations and leverages them for control. We show that the resulting options simultaneously improve exploration, accelerate credit assignment, and generalise effectively to unseen settings, enabling substantially faster learning of complex policies. Wayfarer achieves state-of-the-art performance among single-stream agents on the most challenging Atari 2600 games, with the largest gains in games that require long-horizon exploration and strategic behaviour, such as Montezuma's Revenge and Private Eye.
