arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.11375math.STcs.NAmath.NAstat.MLstat.TH
收藏

多层参数估计的经验Bayes与核流的相合性

Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation

Yifan Chen, Houman Owhadi, Andrew M. Stuart

摘要:

高斯过程回归在统计、机器学习和逆问题中被证明是非常强大的。这种方法成功的一个关键方面,在复杂和现实世界的问题的广泛应用,是分层建模和学习的超参数。本文的目的是研究两种范式的学习层次参数:一个是从概率贝叶斯的角度来看,特别是经验贝叶斯方法,已被大量使用的贝叶斯统计;另一个是从确定性和近似理论的观点,特别是核流算法,最近提出的机器学习文献。本文针对环面上的Matérn类模型,分析了它们在大数据极限下的一致性,以及它们在参数学习中的隐式偏差.一个特殊的技术挑战,我们克服的是学习的规律性参数在Matérn类领域,一致性的结果一直非常罕见的空间统计文献。此外,我们进行了广泛的数值实验超越Matérn模型,进一步比较这两种算法。这些实验展示了其他层次参数的学习,如幅度和长度尺度;它们还说明了模型误指定的设置,其中内核流方法可以显示出优于更传统的经验贝叶斯方法的性能。

英文摘要:

Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world problems, is hierarchical modeling and learning of hyperparameters. The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature. Analysis of their consistency in the large data limit, as well as explicit identification of their implicit bias in parameter learning, are established in this paper for a Matérn-like model on the torus. A particular technical challenge we overcome is the learning of the regularity parameter in the Matérn-like field, for which consistency results have been very scarce in the spatial statistics literature. Moreover, we conduct extensive numerical experiments beyond the Matérn-like model, comparing the two algorithms further. These experiments demonstrate learning of other hierarchical parameters, such as amplitude and lengthscale; they also illustrate the setting of model misspecification in which the kernel flow approach could show superior performance to the more traditional empirical Bayes approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑