arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1401.3870cs.LGcs.AIstat.ML
收藏

学习在部分可观察环境中进行预测,而无需生成模型

Learning to Make Predictions In Partially Observable Environments Without a Generative Model

Franklin and Marshall College(富兰克林与马歇尔学院);University of Michigan(密歇根大学)

Erik Talvitie, Satinder Singh

摘要:

当面对学习高维环境的模型的问题时,常见的方法是限制模型仅进行有限的预测集,从而简化学习问题。这些部分模型可以直接用于决策,或者可以组合在一起以形成更完整的结构化模型。然而,在部分可观察(非马尔可夫)环境中,标准模型学习方法学习生成模型,即在所有可能的未来(如POMDP)上提供概率分布的模型。限制这些模型只做出某些预测并不简单,这样做并不总是简化学习问题。在本文中,我们提出了预测轮廓模型:非生成部分模型的部分可观察系统,使只有一组给定的预测,因此在某些情况下,远比生成模型简单。我们将学习预测配置文件模型的问题形式化为原始模型学习问题的转换,并根据经验表明,即使在标准生成模型过于复杂的系统中,也可以学习预测配置文件模型,这些模型可以做出一小部分重要的预测。

英文摘要:

When faced with the problem of learning a model of a high-dimensional environment, a common approach is to limit the model to make only a restricted set of predictions, thereby simplifying the learning problem. These partial models may be directly useful for making decisions or may be combined together to form a more complete, structured model. However, in partially observable (non-Markov) environments, standard model-learning methods learn generative models, i.e. models that provide a probability distribution over all possible futures (such as POMDPs). It is not straightforward to restrict such models to make only certain predictions, and doing so does not always simplify the learning problem. In this paper we present prediction profile models: non-generative partial models for partially observable systems that make only a given set of predictions, and are therefore far simpler than generative models in some cases. We formalize the problem of learning a prediction profile model as a transformation of the original model-learning problem, and show empirically that one can learn prediction profile models that make a small set of important predictions even in systems that are too complex for standard generative models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑