arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.06303cs.LG
收藏

复杂行动空间中的学习与规划

Learning and Planning in Complex Action Spaces

DeepMind

Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Mohammadamin Barekatain, Simon Schmitt, David Silver

摘要:

许多重要的现实世界的问题有行动空间是高维的,连续的或两者兼而有之,使所有可能的行动完全枚举不可行。相反,只有一小部分的行动可以抽样的政策评估和改进的目的。在本文中,我们提出了一个一般的框架,原因在原则上的政策评估和改进,这样的采样行动子集。这种基于样本的策略迭代框架原则上可以应用于任何基于策略迭代的强化学习算法。具体来说,我们提出了采样MuZero,MuZero算法的扩展,能够通过规划采样动作在具有任意复杂动作空间的域中学习。我们在经典的围棋棋盘游戏和两个连续控制基准域上展示了这种方法:DeepMind Control Suite和Real-World RL Suite。

英文摘要:

Many important real-world problems have action spaces that are high-dimensional, continuous or both, making full enumeration of all possible actions infeasible. Instead, only small subsets of actions can be sampled for the purpose of policy evaluation and improvement. In this paper, we propose a general framework to reason in a principled way about policy evaluation and improvement over such sampled action subsets. This sample-based policy iteration framework can in principle be applied to any reinforcement learning algorithm based upon policy iteration. Concretely, we propose Sampled MuZero, an extension of the MuZero algorithm that is able to learn in domains with arbitrarily complex action spaces by planning over sampled actions. We demonstrate this approach on the classical board game of Go and on two continuous control benchmark domains: DeepMind Control Suite and Real-World RL Suite.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑