arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1811.05965cs.LGcs.PLstat.COstat.ML
收藏

用概率程序组合器组合建模和推理操作

Composing Modeling and Inference Operations with Probabilistic Program Combinators

Northeastern University(东北大学);University of Cambridge(剑桥大学);MPI Tübingen(图宾根马克斯·普朗克研究所)

Eli Sennesh, Adam Ścibior, Hao Wu, Jan-Willem van de Meent

摘要:

具有动态计算图的概率程序可以定义具有无界维数的样本空间上的测度,这构成了贝叶斯非参数化的程序模拟。由于这一模型类的普遍性,推理依赖于“黑盒”蒙特卡罗方法,这些方法往往无法利用条件独立性和交换性,而这在历史上一直是有效推理的基石。我们在这里寻求开发一个“中间地带”之间的概率模型与完全动态和完全静态的计算图。为此,我们介绍了一个组合子库的Probably火炬框架。组合器是接受模型并返回转换后的模型的函数。我们假设模型是动态的,但是模型组合是静态的,在这个意义上,组合子应用发生在对数据评估模型之前。组合器为模型和推理组合提供原语。模型组合子采用经典的函数式编程构造的形式,如map和reduce。这些构造在粗化的表示级别上定义了计算图,其中节点对应于模型,而不是单个变量。推理组合子实现诸如重要性恢复和转换内核的应用等操作,这些操作在保留适当权重的同时改变模型的评估策略。由于这个属性,使用组合子定义的模型可以使用随机方法进行训练,这些方法可以优化变分或唤醒-睡眠风格的目标。作为这一原则的验证,我们使用组合子来实现隐马尔可夫模型的黑盒推理。

英文摘要:

Probabilistic programs with dynamic computation graphs can define measures over sample spaces with unbounded dimensionality, which constitute programmatic analogues to Bayesian nonparametrics. Owing to the generality of this model class, inference relies on `black-box' Monte Carlo methods that are often not able to take advantage of conditional independence and exchangeability, which have historically been the cornerstones of efficient inference. We here seek to develop a `middle ground' between probabilistic models with fully dynamic and fully static computation graphs. To this end, we introduce a combinator library for the Probabilistic Torch framework. Combinators are functions that accept models and return transformed models. We assume that models are dynamic, but that model composition is static, in the sense that combinator application takes place prior to evaluating the model on data. Combinators provide primitives for both model and inference composition. Model combinators take the form of classic functional programming constructs such as map and reduce. These constructs define a computation graph at a coarsened level of representation, in which nodes correspond to models, rather than individual variables. Inference combinators implement operations such as importance resampling and application of a transition kernel, which alter the evaluation strategy for a model whilst preserving proper weighting. Owing to this property, models defined using combinators can be trained using stochastic methods that optimize either variational or wake-sleep style objectives. As a validation of this principle, we use combinators to implement black box inference for hidden Markov models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑