arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1401.3847cs.AI
收藏

概率规划中Bellman误差特征的自动归纳

Automatic Induction of Bellman-Error Features for Probabilistic Planning

Purdue University(普渡大学)

Jia-Hong Wu, Robert Givan

摘要:

领域特定的特征在整个机器学习和决策理论规划中表示问题结构是很重要的。在规划中,一旦提供了状态特征,诸如近似值迭代之类的独立于域的算法就可以学习那些特征的加权组合,这些特征通常表现得很好,作为状态值的启发式估计(例如,目标的距离)。在现实世界领域中成功的应用程序通常需要由人类专家精心制作的功能。在这里,我们提出了学习有用的特定领域的功能集,很少或没有人为干预的自动化过程。我们的方法选择和添加功能,描述状态空间区域的高度不一致的贝尔曼方程(状态贝尔曼错误)在近似值迭代。我们的方法可以使用任何实值特征假设空间和相应的学习方法来从状态值对的训练集中选择特征。我们评估的方法与假设空间定义的关系和命题特征语言,使用九个概率规划域。我们表明,近似值迭代使用的关系特征空间执行在最先进的领域独立的随机关系规划。我们的方法提供了第一个独立于域的方法,成功地玩俄罗斯方块(没有人类工程的功能)。

英文摘要:

Domain-specific features are important in representing problem structure throughout machine learning and decision-theoretic planning. In planning, once state features are provided, domain-independent algorithms such as approximate value iteration can learn weighted combinations of those features that often perform well as heuristic estimates of state value (e.g., distance to the goal). Successful applications in real-world domains often require features crafted by human experts. Here, we propose automatic processes for learning useful domain-specific feature sets with little or no human intervention. Our methods select and add features that describe state-space regions of high inconsistency in the Bellman equation (statewise Bellman error) during approximate value iteration. Our method can be applied using any real-valued-feature hypothesis space and corresponding learning method for selecting features from training sets of state-value pairs. We evaluate the method with hypothesis spaces defined by both relational and propositional feature languages, using nine probabilistic planning domains. We show that approximate value iteration using a relational feature space performs at the state-of-the-art in domain-independent stochastic relational planning. Our method provides the first domain-independent approach that plays Tetris successfully (without human-engineered features).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑