arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1906.02457cs.LGcs.AIstat.ML
收藏

离散强化学习

Clustered Reinforcement Learning

Nanjing University(南京大学)

Xiao Ma, Shen-Yi Zhao, Wu-Jun Li

摘要:

探索策略的设计是强化学习~(RL)中具有挑战性的问题之一,特别是当环境包含大的状态空间或稀疏的奖励时。在探索过程中,Agent试图发现新的区域或高回报质量区域。现有的方法大多没有很好地利用邻近区域的新颖性和质量的当前状态来指导智能体的探索。为了解决这个问题,我们提出了一个新的RL框架,称为\underline{c}聚类\underline{r}嵌入式\underline{l} earning~(CRL),在RL的有效探索。CRL采用聚类的方法将收集到的状态划分为若干类,并在此基础上给予Agent反映当前状态相邻区域类中新颖性和质量的奖励。在一个连续控制任务和几个\emph{Atari 2600}游戏上的实验表明,CRL在大多数情况下都能达到最佳性能。

英文摘要:

Exploration strategy design is one of the challenging problems in reinforcement learning~(RL), especially when the environment contains a large state space or sparse rewards. During exploration, the agent tries to discover novel areas or high reward~(quality) areas. In most existing methods, the novelty and quality in the neighboring area of the current state are not well utilized to guide the exploration of the agent. To tackle this problem, we propose a novel RL framework, called \underline{c}lustered \underline{r}einforcement \underline{l}earning~(CRL), for efficient exploration in RL. CRL adopts clustering to divide the collected states into several clusters, based on which a bonus reward reflecting both novelty and quality in the neighboring area~(cluster) of the current state is given to the agent. Experiments on a continuous control task and several \emph{Atari 2600} games show that CRL can outperform other state-of-the-art methods to achieve the best performance in most cases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑