arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

基于高斯重拟合的核岭回归预测误差上置信界

Upper Confidence Bounds for the Prediction Error of Kernel Ridge Regression via Gaussian Refitting

Yijin Ni, Xiaoming Huo

arXiv 2607.28846首次发表:更新:

AI 中文总结

该研究针对核岭回归预测误差,提出高斯重拟合方法得到上置信界,其收缩速率符合极小极大理论,实证覆盖性优于交叉验证,还可扩展到非线性约束估计器与空间数据。

AI 中文摘要

评估单个模型拟合效果需要一个可计算的上置信界,以衡量拟合结果与未知真实值之间的差距,因为均值估计会忽略实现方差。标准交叉验证的边际因噪声波动被限制在$n^{-1/2}$量级,即使真实误差收缩得更快。尽管野生重拟合可消除该噪声水平,但现有的拉德马赫符号方法对核岭回归会退化,且依赖不可观测的量。我们提出针对核岭回归的高斯重拟合方法。根据安德森不等式,拟合的移动幅度随噪声大小单调变化,从而产生可计算的尾界。仅假设噪声对称,该界无需矩假设,且可通过顺序统计量在任意置信水平下校准。理论上,使用最坏情况包络时,该界以极小极大速率$O_P(n^{-2s/(2s+1)})$收缩,与预测误差匹配。实证上,使用实用的数据驱动包络时,该界在真实95%误差分位数的两倍内保持完全覆盖,相比之下,交叉验证超出该分位数的倍数最高达51,在无限方差噪声下甚至超出数百倍。该方法在实证上可扩展到非线性约束估计器和真实空间数据。

英文摘要

Assessing a single model fit requires a computable upper confidence bound for the gap between the fit and the unknown truth, as mean estimates ignore realization variance. Standard cross-validation margins are bottlenecked at order $n^{-1/2}$ by noise fluctuations, even when the true error shrinks faster. While wild refitting cancels this noise level, existing Rademacher sign methods degenerate for kernel ridge regression and rely on unobservable quantities. We propose a Gaussian refit for kernel ridge regression. By Anderson's inequality, the fit movement is monotone in the noise sizes, yielding a computable tail bound. Assuming only symmetric noise, the bound requires no moment assumptions and is calibrated at any confidence level via order statistics. Theoretically, using a worst-case envelope, the bound contracts at the minimax rate $O_P(n^{-2s/(2s+1)})$, correctly matching the prediction error. Empirically, using a practical data-driven envelope, the bound maintains full coverage within twice the true $95\%$ error quantile. By contrast, cross-validation exceeds this quantile by factors up to $51$, and by hundreds under infinite-variance noise. The procedure extends empirically to nonlinear constrained estimators and real spatial data.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑