arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1606.05174cs.AI
收藏

深度强化学习发现内部模型

Deep Reinforcement Learning Discovers Internal Models

Technion(以色列理工学院)

Nir Baram, Tom Zahavy, Shie Mannor

摘要:

深度强化学习(Deep Reinforcement Learning,DRL)是一个热门的研究领域,在诸如玩雅达利游戏、解决围棋和控制机器人等具有挑战性的问题上显示出巨大的前景。虽然DRL代理在实践中表现良好,但我们仍然缺乏分析其性能的工具。在这项工作中,我们提出了半聚合MDP(SAMDP)模型。一种最适合描述同时展现空间和时间层次的策略的模型。我们描述了它的优势,用于分析训练的政策比其他建模方法,并表明,在正确的状态表示,如DQN代理,SAMDP可以帮助识别技能。我们详细介绍了从记录的轨迹创建它的自动过程,直到在t-SNE地图上呈现它。我们解释了如何评估其健身和令人惊讶的结果表明高兼容性的政策在手。最后,我们展示了如何使用SAMDP模型,可以从代理中挤出额外的性能增益。

英文摘要:

Deep Reinforcement Learning (DRL) is a trending field of research, showing great promise in challenging problems such as playing Atari, solving Go and controlling robots. While DRL agents perform well in practice we are still lacking the tools to analayze their performance. In this work we present the Semi-Aggregated MDP (SAMDP) model. A model best suited to describe policies exhibiting both spatial and temporal hierarchies. We describe its advantages for analyzing trained policies over other modeling approaches, and show that under the right state representation, like that of DQN agents, SAMDP can help to identify skills. We detail the automatic process of creating it from recorded trajectories, up to presenting it on t-SNE maps. We explain how to evaluate its fitness and show surprising results indicating high compatibility with the policy at hand. We conclude by showing how using the SAMDP model, an extra performance gain can be squeezed from the agent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑