发表机构
Inha University（仁荷大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究探讨贝叶斯在线学习中近似后验的快速速率，证明满足精度要求的近似后验方法可继承精确贝叶斯预测的快速遗憾界，并通过三个实例验证了相关方法的有效性。
AI 中文摘要
精确贝叶斯预测具有快速的预测遗憾保证，但精确后验的更新或表示对于在线使用而言可能成本过高。我们研究计算近似在何种情况下能保留这些统计保证。我们表明，后验近似的累积代价可由精确Gibbs后验的收缩半径与近似后验和精确后验之间的Wasserstein距离的相互作用来控制。我们的一般定理表明，只要精确贝叶斯预测能达到快速遗憾界，任何以足够精度跟踪精确后验的近似后验方法都会继承相同的快速遗憾，仅存在一个由近似误差决定的加性项。我们开发了三个在线学习示例：对于具有强凸正则化损失的线性模型，投影Langevin算法产生的近似后验能达到对数遗憾；对于Sobolev椭球上的无限维典型指数族序列模型，保留先验的截断方法以次线性内存和每个观测的恒定更新成本达到极小极大预测遗憾速率；对于随机设计高斯过程（GP）回归，带有诱导变量的稀疏变分后验能达到与精确GP相同的预测遗憾速率，但计算成本显著更低。
英文摘要
Exact Bayes prediction enjoys fast predictive regret guarantees, but exact posterior updating or representation may be too costly for online use. We study when these statistical guarantees are preserved by computational approximations. We show that the cumulative price of posterior approximation can be governed by the interaction between the contraction radius of the exact Gibbs posterior and the Wasserstein distance between the approximate and exact posteriors. Our general theorem shows that whenever exact Bayes prediction achieves a fast regret bound, any approximate posterior method that tracks the exact posterior with sufficient accuracy inherits the same fast regret, up to an additive term determined by the approximation error. Three online learning examples are developed. For linear models with strongly convex regularized losses, a projected Langevin algorithm yields an approximate posterior that achieves logarithmic regret. For an infinite-dimensional canonical exponential family sequence model over a Sobolev ellipsoid, a prior-preserving truncation method attains the minimax predictive regret rate with sublinear memory and constant update cost per observation. For random-design Gaussian process (GP) regression, a sparse variational posterior with inducing variables achieves the same predictive regret rate as the exact GP, but at substantially lower computational cost.
