离线目标条件强化学习中的抽象
Abstraction for Offline Goal-Conditioned Reinforcement Learning
AI总结：
本文提出了一种在离线目标条件强化学习中利用抽象的方法，通过引入相对化选项和不同层次的表示，提高了在相似状态空间上下文中的经验复用能力，从而提升了性能。
摘要：
马尔可夫决策过程（MDPs）在现实中的目标条件强化学习（GCRL）中往往由于对称性和状态-目标对之间的共享结构而表现出显著的冗余性。虽然分层策略已被提出以通过时间抽象减少时间跨度来改进离线GCRL，但本文证明层次结构也能够实现绝对抽象。通过引入相对化选项以及为不同层次的层次结构引入不同的表示，我们展示了智能体如何在相似的状态空间上下文中重用经验。基于这一框架，我们介绍了两种简单的算法用于学习相对化选项和从绝对参考框架中抽象。我们的实验表明，这种归纳偏置在离线GCRL中显著提高了性能。
英文摘要：
Markov Decision Processes (MDPs) often exhibit significant redundancy due to symmetries and shared structure across state-goal pairs in real-world Goal-Conditioned Reinforcement Learning (GCRL). While hierarchical policies have been motivated for horizon reduction via temporal abstraction in offline GCRL, we demonstrate that hierarchy also enables absolute abstraction. By introducing relativised options as well as distinct representations for different levels of the hierarchy, we demonstrate how an agent can reuse experience across similar contexts of the state-space. Based on this framework, we introduce two simple algorithms for learning relativised options and abstracting from the absolute frame of reference. Our experiments show that such inductive biases significantly improve performance in offline GCRL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
