arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1904.02435cs.LGcs.AIcs.NE
收藏

自适应的目标允许将预测模型转移到新任务

Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks

University of Oslo(奥斯陆大学)

Kai Olav Ellefsen, Jim Torresen

摘要:

强化学习的一个长期挑战是使智能体能够学习其环境的模型,该模型可以被转移到具有相同底层规则的世界中解决其他问题。这很困难的一个原因是学习环境的准确模型的挑战。如果这样的模型是不准确的,代理人的计划和行动将可能是次优的,并可能导致错误的结果。基于模型的强化学习的最新进展提高了智能体学习和使用预测模型的能力。在本文中,我们扩展了最近的深度学习架构,该架构学习环境的预测模型,该模型旨在仅预测几个关键测量值,这些测量值指示代理的性能。仅预测环境的几个测量值而不是整个未来状态,使得学习有价值的预测模型更加可行。我们用一个小的、不断发展的神经网络来扩展这个预测模型,它可以建议在当前状态下追求的最佳目标。我们证明,这使得预测模型转移到新的情况下,目标是不同的,自适应目标甚至可以在线调整代理的行为,改变其策略,以适应当前的情况。

英文摘要:

A long-standing challenge in Reinforcement Learning is enabling agents to learn a model of their environment which can be transferred to solve other problems in a world with the same underlying rules. One reason this is difficult is the challenge of learning accurate models of an environment. If such a model is inaccurate, the agent's plans and actions will likely be sub-optimal, and likely lead to the wrong outcomes. Recent progress in model-based reinforcement learning has improved the ability for agents to learn and use predictive models. In this paper, we extend a recent deep learning architecture which learns a predictive model of the environment that aims to predict only the value of a few key measurements, which are be indicative of an agent's performance. Predicting only a few measurements rather than the entire future state of an environment makes it more feasible to learn a valuable predictive model. We extend this predictive model with a small, evolving neural network that suggests the best goals to pursue in the current state. We demonstrate that this allows the predictive model to transfer to new scenarios where goals are different, and that the adaptive goals can even adjust agent behavior on-line, changing its strategy to fit the current context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑