arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.28256stat.MLcs.LG
收藏

面向稀疏变分高斯过程的几何感知块选择

Geometry-Aware Block Selection for Sparse Variational Gaussian Processes

Nanyang Technological University(南洋理工大学)

Matteo Hoxha

AI总结:

本文提出几何感知块选择算法,结合双重随机优化框架,改进稀疏变分高斯过程,在固定诱导点预算下降低测试RMSE和NLL,还探索用替代条件后验结构优化超参数与诱导点。

摘要:

稀疏变分高斯过程(SVGPs)是一种广泛应用的、原理性的可扩展高斯过程建模方法,其成功源于理论上可解释的结构:它为完整高斯过程的边际似然提供下界,同时支持可扩展的随机优化。近期研究探索了后验条件分布的修改,表明这类扩展可改进这一成熟框架,尤其是引入块对角结构的缩放矩阵能得到更紧的高斯过程边际似然下界。本文研究在固定诱导点预算下,选择能收紧变分目标的块是否可降低测试均方根误差(RMSE)和负对数似然(NLL)。由于缩放矩阵通过完整残差协方差矩阵对数行列式的上界对变分下界产生贡献,本文的贡献如下:第一,提出一种简单、可扩展的几何感知算法,具有明确的计算复杂度和低计算开销,通过选择合适的数据划分来恢复大部分对数行列式贡献;第二,展示如何使用双重随机优化框架训练这些模型;第三,研究基于替代条件后验结构的稀疏高斯过程方法如何用于学习SVGPs的更优超参数和诱导点。

英文摘要:

Sparse variational Gaussian processes (SVGPs) are a widely used, principled approach to scalable Gaussian process modelling. Their success stems from their theoretically interpretable structure: they provide a lower bound on the marginal likelihood of the full GP while allowing scal- able stochastic optimization. Recent work has explored modifications of the posterior conditional distribution and has shown that such extensions can improve upon this well-established framework. In particular, introducing a block-diagonal structured scal- ing matrix leads to a tighter lower bound on the GP marginal likelihood. We investigate whether selecting blocks that tighten the variational objective improves test RMSE and NLL at a fixed inducing-point budget. Since the scaling matrix contributes to the variational lower bound through an upper bound on the log-determinant of the full residual covariance matrix, our work makes the following contributions. First, we introduce a simple, scalable, geometry-aware algorithm, with explicit computational complexity and low computational overhead, that recovers a large fraction of this log- determinant contribution by selecting an appropriate partition of the data. Second, we show how to train these models using a doubly stochastic optimization framework. Finally, we investigate how sparse GP methods based on alternative conditional posterior structures can be used to learn better hyperparameters and inducing points for SVGP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑