在线优惠券收集器问题及其在终身强化学习中的应用
The Online Coupon-Collector Problem and Its Application to Lifelong Reinforcement Learning
摘要：
在一系列相关任务之间传递知识是强化学习（RL）的一个重要挑战。尽管有许多令人鼓舞的经验证据，但很少有理论分析。在本文中，我们研究了一类终身强化学习问题：代理解决一系列的任务建模为有限的马尔可夫决策过程（MDP），其中每一个是从一个有限的MDP集相同的状态/动作集和不同的过渡/奖励功能。出于终身学习中跨任务探索的需要，我们提出了一个新的在线优惠券收集器问题，并给出了一个最优算法。这使我们能够开发一种新的终身强化学习算法，即使任务序列是由对手生成的，其任务序列的总体样本复杂度也比单任务学习小得多。该算法的好处在模拟问题，包括最近推出的人机交互问题。
英文摘要：
Transferring knowledge across a sequence of related tasks is an important challenge in reinforcement learning (RL). Despite much encouraging empirical evidence, there has been little theoretical analysis. In this paper, we study a class of lifelong RL problems: the agent solves a sequence of tasks modeled as finite Markov decision processes (MDPs), each of which is from a finite set of MDPs with the same state/action sets and different transition/reward functions. Motivated by the need for cross-task exploration in lifelong learning, we formulate a novel online coupon-collector problem and give an optimal algorithm. This allows us to develop a new lifelong RL algorithm, whose overall sample complexity in a sequence of tasks is much smaller than single-task learning, even if the sequence of tasks is generated by an adversary. Benefits of the algorithm are demonstrated in simulated problems, including a recently introduced human-robot interaction problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
