arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1206.6473cs.AIcs.LG
收藏

基于最优期权模型的组合规划

Compositional Planning Using Optimal Option Models

University College London(伦敦大学学院)

David Silver, Kamil Ciosek

摘要:

在本文中,我们介绍了一个框架的期权模型组成。期权模型是时间抽象,就像经典规划中的宏操作符一样,直接从开始状态跳到结束状态。以前的工作主要集中在构建期权模型的原始行动,通过内部的选择权模型学习,或使用期权模型来构建一个价值函数,通过内部的选择权规划。我们提出了一个统一的观点内和间的选择模型学习,贝尔曼方程的主要概括的基础上。我们的基本操作是将期权模型递归组合成其他期权模型。这一关键思想支持在多个抽象级别上进行组合规划。我们说明了我们的框架,使用动态规划算法,同时构建多个子目标的最优选项模型,并搜索这些选项模型,以提供快速的进展,对其他子目标。

英文摘要:

In this paper we introduce a framework for option model composition. Option models are temporal abstractions that, like macro-operators in classical planning, jump directly from a start state to an end state. Prior work has focused on constructing option models from primitive actions, by intra-option model learning; or on using option models to construct a value function, by inter-option planning. We present a unified view of intra- and inter-option model learning, based on a major generalisation of the Bellman equation. Our fundamental operation is the recursive composition of option models into other option models. This key idea enables compositional planning over many levels of abstraction. We illustrate our framework using a dynamic programming algorithm that simultaneously constructs optimal option models for multiple subgoals, and also searches over those option models to provide rapid progress towards other subgoals.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑