arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.08028cs.LG
收藏

关于元强化学习的教程

A Tutorial on Meta-Reinforcement Learning

University of Oxford(牛津大学);Stanford University(斯坦福大学)

Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, Shimon Whiteson

AI总结:

本文是一篇关于元强化学习的教程,详细描述了其问题设置及变体,基于任务分布和学习预算对算法进行分类综述,并探讨了该领域的开放问题。

摘要:

尽管深度强化学习(RL)推动了机器学习中多个备受瞩目的成功,但由于其通常较差的数据效率及其生成策略的有限泛化能力,阻碍了它更广泛的应用。缓解这些局限性的一种有前景的方法是,在一个称为元RL的过程中,将开发更好的RL算法本身作为一个机器学习问题。元RL最常在以下问题设置中被研究:给定一个任务分布,目标是学习一种策略,该策略能够利用尽可能少的数据适应任务分布中的任何新任务。在本调查中,我们详细描述了元RL问题设置及其主要变体。我们讨论了在宏观层面上,如何根据是否存在任务分布以及每个单独任务可用的学习预算,对元RL研究进行聚类。利用这些聚类,我们随后调查了元RL算法和应用。最后,我们提出了在使元RL成为深度RL从业者标准工具箱一部分的道路上所面临的开放问题。

英文摘要:

While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficiency and the limited generality of the policies it produces. A promising approach for alleviating these limitations is to cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible. In this survey, we describe the meta-RL problem setting in detail as well as its major variations. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. Using these clusters, we then survey meta-RL algorithms and applications. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑