arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1812.01129cs.LGcs.AI
收藏

基于模型的强化学习中规划器过拟合的抑制

Mitigating Planner Overfitting in Model-Based Reinforcement Learning

Stanford University(斯坦福大学);Brown University(布朗大学);University of Michigan(密歇根大学)

Dilip Arumugam, David Abel, Kavosh Asadi, Nakul Gopalan, Christopher Grimm, Jun Ki Lee, Lucas Lehnert, Michael L. Littman

摘要:

具有不准确的环境模型的智能体面临着一个困难的选择:它可以忽略模型中的错误,并以它确定的相对于模型最优的任何方式在现实世界中行动。或者,它可以采取更保守的立场,避开其模型,只通过现实世界的交互来优化其行为。后一种方法从经验中学习可能非常缓慢,而前一种方法可能导致“规划者过度拟合”-代理行为的各个方面被优化以利用其模型中的错误。本文探讨了一个中间位置,规划者试图通过一种正则化的计划,它认为,以避免过度拟合。我们提出了三种不同的方法,可以证明减轻规划过度拟合的学习环境。

英文摘要:

An agent with an inaccurate model of its environment faces a difficult choice: it can ignore the errors in its model and act in the real world in whatever way it determines is optimal with respect to its model. Alternatively, it can take a more conservative stance and eschew its model in favor of optimizing its behavior solely via real-world interaction. This latter approach can be exceedingly slow to learn from experience, while the former can lead to "planner overfitting" - aspects of the agent's behavior are optimized to exploit errors in its model. This paper explores an intermediate position in which the planner seeks to avoid overfitting through a kind of regularization of the plans it considers. We present three different approaches that demonstrably mitigate planner overfitting in reinforcement-learning environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑