arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1108.3614cs.AIcs.RO
收藏

强化学习在实践中的特点

Feature Reinforcement Learning In Practice

Research School of Computer Science, CECS, ANU(澳大利亚国立大学计算机与工程学院计算机科学研究学院)

Phuong Nguyen, Peter Sunehag, Marcus Hutter

摘要:

在最近使用基于历史的方法来解决强化学习中的感知混淆的浪潮之后,我们引入了一种基于特征强化学习框架的算法,称为PhiMDP。为了创建一个实用的算法,我们设计了一个随机搜索过程的一类上下文树的基础上并行回火和一个专门的建议分布。我们为PhiMDP提供了第一个经验评估。我们提出的算法实现了优于经典的U树算法和最近的主动LZ算法的性能,并与MC-AIXI-CTW竞争,保持一个basternity混合在所有上下文树到选定的depth.We鼓励我们的能力,与这种复杂的方法使用一个算法,简单地选择一个单一的模型,并使用相应的MDP Q学习。我们的PhiMDP算法要简单得多,但消耗的时间和内存更少。这些结果表明,我们未来的工作,攻击更复杂和更大的问题的承诺。

英文摘要:

Following a recent surge in using history-based methods for resolving perceptual aliasing in reinforcement learning, we introduce an algorithm based on the feature reinforcement learning framework called PhiMDP. To create a practical algorithm we devise a stochastic search procedure for a class of context trees based on parallel tempering and a specialized proposal distribution. We provide the first empirical evaluation for PhiMDP. Our proposed algorithm achieves superior performance to the classical U-tree algorithm and the recent active-LZ algorithm, and is competitive with MC-AIXI-CTW that maintains a bayesian mixture over all context trees up to a chosen depth.We are encouraged by our ability to compete with this sophisticated method using an algorithm that simply picks one single model, and uses Q-learning on the corresponding MDP. Our PhiMDP algorithm is much simpler, yet consumes less time and memory. These results show promise for our future work on attacking more complex and larger problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑