arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.05079cs.LGcs.AI
收藏

SAGE:在深度强化学习中为近视模型生成符号目标

SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement Learning

RMIT University(皇家墨尔本理工大学)

Andrew Chester, Michael Dann, Fabio Zambetta, John Thangarajah

摘要:

基于模型的强化学习算法通常比无模型算法更有效,特别是在稀疏奖励问题中。不幸的是,许多有趣的领域过于复杂,无法指定传统的基于模型的方法所需的完整模型。学习模型需要大量的环境样本,如果环境很难探索,可能无法捕获关键信息。如果我们能够指定一个不完整的模型,并允许代理学习如何最好地使用它,我们就可以利用我们对许多领域的部分理解。现有的混合规划和学习系统,解决这个问题,往往施加高度限制性的假设上的各种模型,可以使用,限制其适用范围广泛的领域。在这项工作中,我们提出了SAGE,学习和规划相结合的算法,利用以前无法使用的不完整的模型类。它以一种新颖的方式结合了符号规划和神经学习方法的优势,在出租车世界和Minecraft的变化中优于竞争方法。

英文摘要:

Model-based reinforcement learning algorithms are typically more sample efficient than their model-free counterparts, especially in sparse reward problems. Unfortunately, many interesting domains are too complex to specify the complete models required by traditional model-based approaches. Learning a model takes a large number of environment samples, and may not capture critical information if the environment is hard to explore. If we could specify an incomplete model and allow the agent to learn how best to use it, we could take advantage of our partial understanding of many domains. Existing hybrid planning and learning systems which address this problem often impose highly restrictive assumptions on the sorts of models which can be used, limiting their applicability to a wide range of domains. In this work we propose SAGE, an algorithm combining learning and planning to exploit a previously unusable class of incomplete models. This combines the strengths of symbolic planning and neural learning approaches in a novel way that outperforms competing methods on variations of taxi world and Minecraft.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑