arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.01074math.STcs.NAmath.NAstat.TH
收藏

平坦极限下的高斯过程回归

Gaussian Process Regression in the Flat Limit

Simon Barthelmé, Pierre-Olivier Amblard, Nicolas Tremblay, Konstantin Usevich

AI总结:

本文研究了高斯过程回归在平坦极限(核函数长度尺度趋于无穷大)下的理论行为,发现其预测均值和方差会趋向于多项式或样条回归,揭示了现有数值方法难以观测到的最优预测区间。

摘要:

高斯过程(GP)回归是贝叶斯统计中的一个基本工具。它也被称为kriging,是频率学派核岭回归的贝叶斯对应物。大多数关于GP回归的理论工作都集中在大n渐近分析上,即随着数据量的增加。在简单情况(如规则网格上的位置)之外,固定样本分析要困难得多。在这项工作中,我们进行了一项固定样本分析,该分析最初由Driscoll & Fornberg (2002)在逼近论的背景下进行了研究,被称为“平坦极限”。在平坦极限渐近分析中,目标是当核函数的长度尺度趋于无穷大时表征核方法,使得核在数据范围内显得平坦。令人惊讶的是,这个极限是明确定义的,并表现出有趣的行为:Driscoll & Fornberg表明,如果核是高斯核,径向基插值在平坦极限下会收敛到多项式插值。后续工作表明,这在多变量设置中也成立,但高斯核以外的核可能以(多调和)样条作为极限插值函数。利用核矩阵在平坦极限下谱行为的最新结果,我们研究了高斯过程回归的平坦极限。结果表明,高斯过程回归在平坦极限下趋向于(多变量)多项式回归,或(多调和)样条回归,具体取决于核。重要的是,这对预测均值和预测方差都成立,因此后验预测分布变得等价。我们的结果具有实际意义:例如,它们表明在留一法(leave-one-out)成本意义上最优的GP预测可能出现在非常大的长度尺度下,但由于数值困难,这对当前方法来说是不可见的。

英文摘要:

Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-$n$ asymptotics, i.e. as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the ``flat limit''. In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Subsequent work showed that this holds true in the multivariate setting as well, but that kernels other than the Gaussian may have (polyharmonic) splines as the limit interpolant. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑