arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1812.04403stat.MLcs.LGstat.CO
收藏

在似然结构中编码先验知识

Encoding prior knowledge in the structure of the likelihood

Max-Planck-Institut für Astrophysik(马克斯·普朗克天体物理研究所);Ludwig-Maximilians-Universität München(慕尼黑大学)

Jakob Knollmüller, Torsten A. Enßlin

摘要:

由于层次结构之间的强依赖性,深层次模型的推理是有问题的。我们研究了基于多元分布变换的模型参数的具体变换。这种变换是重新参数化技巧的一种特殊形式,它简化了层次结构,并导致所有结果参数的标准高斯先验。该变换还将所有先验信息转移到似然的结构中,从而先验地将变换后的参数彼此解耦。在这个标准化空间中的变分高斯近似在相对无信息数据的情况下将是极好的。此外,对数后验的曲率在受数据弱约束的方向上是良好的,允许在这种情况下进行快速推断。在一个例子中,我们显式地对具有先验未知相关结构的高斯过程回归进行变换。深度模型在高度信息化的情况下推断迅速,在信息不充分的情况下推断缓慢。平面模型显示出完全相反的性能模式。深度透视和平面透视的综合提供了它们的综合优势,克服了个体的局限性,从而导致更快的推理。

英文摘要:

The inference of deep hierarchical models is problematic due to strong dependencies between the hierarchies. We investigate a specific transformation of the model parameters based on the multivariate distributional transform. This transformation is a special form of the reparametrization trick, flattens the hierarchy and leads to a standard Gaussian prior on all resulting parameters. The transformation also transfers all the prior information into the structure of the likelihood, hereby decoupling the transformed parameters a priori from each other. A variational Gaussian approximation in this standardized space will be excellent in situations of relatively uninformative data. Additionally, the curvature of the log-posterior is well-conditioned in directions that are weakly constrained by the data, allowing for fast inference in such a scenario. In an example we perform the transformation explicitly for Gaussian process regression with a priori unknown correlation structure. Deep models are inferred rapidly in highly and slowly in poorly informed situations. The flat model show exactly the opposite performance pattern. A synthesis of both, the deep and the flat perspective, provides their combined advantages and overcomes the individual limitations, leading to a faster inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑