通过剩余策略梯度方法利用类抽象进行常识强化学习
Leveraging class abstraction for commonsense reinforcement learning via residual policy gradient methods
摘要：
使强化学习（RL）代理能够利用知识库，同时从经验中学习，有望在知识密集型领域推进RL。然而，事实证明，很难利用不是针对环境手动定制的知识。我们建议使用开源知识图谱中的子类关系来抽象特定对象。我们开发了一个剩余的策略梯度方法，能够在类层次结构中的不同抽象级别的知识集成。我们的方法提高了样本效率，并推广到常识游戏中看不见的对象，但我们也研究了故障模式，如过度的噪声提取类知识或环境中的小类结构。
英文摘要：
Enabling reinforcement learning (RL) agents to leverage a knowledge base while learning from experience promises to advance RL in knowledge intensive domains. However, it has proven difficult to leverage knowledge that is not manually tailored to the environment. We propose to use the subclass relationships present in open-source knowledge graphs to abstract away from specific objects. We develop a residual policy gradient method that is able to integrate knowledge across different abstraction levels in the class hierarchy. Our method results in improved sample efficiency and generalisation to unseen objects in commonsense games, but we also investigate failure modes, such as excessive noise in the extracted class knowledge or environments with little class structure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
