arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

最优恢复遇见贝叶斯学习:最坏情况界何时值得

Optimal Recovery Meets Bayesian Learning: Where Worst-Case Bounds Pay Off

Gordei Verbii

arXiv 2609.29622首次发表:更新:

AI 中文总结

本文统一最坏情况最优恢复与贝叶斯学习,证明Morozov校准在可复现性和后端鲁棒性上优于现有方法,并指出协变量偏移下OR的局限,提出按数据机制匹配保证工具的设计原则。

AI 中文摘要

最坏情况最优恢复(OR)和贝叶斯学习用两种词汇描述相同的高斯-二次-希尔伯特问题。我们强化了对应关系——信息半径等于一个经nugget优化的高斯过程后验方差,并由后验均值在闭式平衡nugget处达到——并在三个已发表的贝叶斯系统中衡量最坏情况一侧的得失。账目是双向的:损失与胜利同样具有指导意义。Morozov校准在$\sigma$-盲规则失效之处,以$1.00$-$1.19\times$的因子追踪测试访问预言机,在噪声抽样间的可复现性高出$4.9$-$6.3\times$($p=0.002$-$0.004$),并且是唯一在更换后端后选择仍然成立的可用规则(对比发布权重、ML-II和GCV的$12$-$30\times$);紧致证书在信息论下限处覆盖,无数值松弛。但在可交换数据上,分裂共形在区间分数上直接胜过OR;水填充先验在没有预言机噪声提示时毫无增益;在协变量偏移下,OR带在每个数据集上保持覆盖率,但在大多数单元上区间分数输给分裂共形和无特征常数带;起作用的不是偏移本身,而是可学习目标上的偏移,这可由一个无需训练即可审计的统计量在任何模型拟合之前预测。在贝叶斯优化中,认证宽度是有效性下限,我们证明其标量膨胀在可检查的边际条件下是惰性的,并在每一步检查。惰性是分级的而非二元的,既体现在膨胀大小上也体现在目标上:$\kappa{=}2$在$\kappa{=}5$惰性的所有地方以及更多单元上惰性,而$\kappa{=}5$移动了一半的Ackley种子和每一个Griewank种子。探索是形状问题而非尺度问题。设计规则:将保证工具与数据机制匹配,并首先审计机制。

英文摘要

Worst-case Optimal Recovery (OR) and Bayesian learning describe the same Gaussian-quadratic-Hilbert problems in two vocabularies. We sharpen the correspondence - the radius of information equals a nugget-optimized GP posterior variance and is attained by the posterior mean at a closed-form balance nugget - and measure, inside three published Bayesian systems, where the worst-case side pays. The ledger is two-sided: the losses instruct as much as the wins. Morozov calibration tracks a test-access oracle within $1.00$-$1.19\times$ where $σ$-blind rules fail, is $4.9$-$6.3\times$ more reproducible across noise draws ($p=0.002$-$0.004$), and is the only deployable rule whose selection survives a change of backend ($1.36\times$ against $12$-$30\times$ for the released weight, ML-II and GCV); tight certificates cover at the information-theoretic floor with no numerical slack. But on exchangeable data split-conformal beats the OR head on interval score, a water-filling prior adds nothing without an oracle noise hint, and under covariate shift the OR band keeps coverage on every dataset yet loses interval score to split-conformal, and to a feature-free constant band, on most cells; what pays is not shift but shift on a learnable target, which a training-free audit statistic predicts before any model is fitted. In Bayesian optimization the certified width is a validity floor whose scalar inflation we prove inert under a checkable margin condition and check at every step. Inertness is graded, not binary, and in the size of the inflation as much as in the objective: $κ{=}2$ is inert wherever $κ{=}5$ is and on more cells besides, while $κ{=}5$ moves half the Ackley seeds and every Griewank seed. Exploration is a shape problem, not a scale one. The design rule: match the guarantee tool to the data regime, and audit the regime first.

Comments48 pages, 22 figures

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑