arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.10801stat.MLcs.LG
收藏

预测复杂性先验

Predictive Complexity Priors

University of Amsterdam(阿姆斯特丹大学);University of Cambridge(剑桥大学)

Eric Nalisnick, Jonathan Gordon, José Miguel Hernández-Lobato

摘要:

众所周知,贝叶斯先验对于诸如神经网络之类的复杂模型是非常困难的。空间的高维性和过度参数化使得关于参数的推理变得具有挑战性。看起来良性和无信息的先验可能对模型的预测产生不直观和有害的影响。出于这个原因,我们提出了预测复杂性先验:通过将模型的预测与参考模型的预测进行比较来定义的函数先验。虽然最初定义在模型输出上,但我们通过改变变量将先验转移到模型参数上。然后,传统的贝叶斯工作流程可以照常进行。我们在高维回归之前应用我们的预测复杂性,对神经网络深度进行推理,并共享Few-Shot学习的统计强度。

英文摘要:

Specifying a Bayesian prior is notoriously difficult for complex models such as neural networks. Reasoning about parameters is made challenging by the high-dimensionality and over-parameterization of the space. Priors that seem benign and uninformative can have unintuitive and detrimental effects on a model's predictions. For this reason, we propose predictive complexity priors: a functional prior that is defined by comparing the model's predictions to those of a reference model. Although originally defined on the model outputs, we transfer the prior to the model parameters via a change of variables. The traditional Bayesian workflow can then proceed as usual. We apply our predictive complexity prior to high-dimensional regression, reasoning over neural network depth, and sharing of statistical strength for few-shot learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑