arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1603.02041cs.AIcs.LG
收藏

多任务强化学习中的共享表示学习

Learning Shared Representations in Multi-task Reinforcement Learning

University College London(伦敦大学学院)

Diana Borsa, Thore Graepel, John Shawe-Taylor

摘要:

我们研究了多任务强化学习(MT-RL)中的一个范例,其中代理被放置在一个环境中,需要学习在这个空间内执行一系列任务。由于环境没有改变,任务之间可能存在很多共同点,学习单独解决它们似乎是非常浪费的。在本文中,我们明确地建模和学习这种共享结构,因为它出现在状态-动作值空间。我们将展示如何通过修改流行的值迭代和策略迭代过程来共同学习最优值函数,以适应这种共享表示假设,并利用多任务监督学习的力量。最后,我们证明了所提出的模型和训练程序,能够推断出良好的价值函数,即使在低样本制度。除了数据效率之外,我们还将在分析中表明,跨任务联合学习状态空间的抽象会导致更强大,可转移的表示,具有更好的泛化潜力。这种共享的表示假设并利用多任务监督学习的能力。最后,我们证明了所提出的模型和训练程序,能够推断出良好的价值函数,即使在低样本制度。除了数据效率之外,我们还将在分析中表明,跨任务联合学习状态空间的抽象会导致更强大,可转移的表示,具有更好的泛化潜力。

英文摘要:

We investigate a paradigm in multi-task reinforcement learning (MT-RL) in which an agent is placed in an environment and needs to learn to perform a series of tasks, within this space. Since the environment does not change, there is potentially a lot of common ground amongst tasks and learning to solve them individually seems extremely wasteful. In this paper, we explicitly model and learn this shared structure as it arises in the state-action value space. We will show how one can jointly learn optimal value-functions by modifying the popular Value-Iteration and Policy-Iteration procedures to accommodate this shared representation assumption and leverage the power of multi-task supervised learning. Finally, we demonstrate that the proposed model and training procedures, are able to infer good value functions, even under low samples regimes. In addition to data efficiency, we will show in our analysis, that learning abstractions of the state space jointly across tasks leads to more robust, transferable representations with the potential for better generalization. this shared representation assumption and leverage the power of multi-task supervised learning. Finally, we demonstrate that the proposed model and training procedures, are able to infer good value functions, even under low samples regimes. In addition to data efficiency, we will show in our analysis, that learning abstractions of the state space jointly across tasks leads to more robust, transferable representations with the potential for better generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑