通过稀疏矩阵表示加性高斯过程
Representing Additive Gaussian Processes by Sparse Matrices
AI总结：
本研究证明加性Matérn高斯过程的后验均值、方差、对数似然及梯度均可由稀疏矩阵/向量表示，将相关计算复杂度降至O(n log n)，并应用于贝叶斯优化大幅降低采集函数等的计算复杂度。
摘要：
在广义加性模型中，加性Matérn高斯过程（GPs）是可扩展高维问题领域最常用的模型之一。得益于其加性结构和随机微分方程表示，基于反向拟合（back-fitting）的算法可将后验均值的计算时间复杂度从$O(n^3)$降至$O(n\backslash log n)$，其中$n$为数据规模。然而，将这类算法推广以高效计算后验方差和最大对数似然仍是一个未解决的问题。本研究证明，对于加性Matérn GPs，不仅后验均值，后验方差、对数似然以及这三个函数的梯度都可以通过仅涉及稀疏矩阵和稀疏向量的公式来表示。我们展示了如何利用这些稀疏公式推广基于反向拟合的算法，以$O(n \backslash log n)$的时间高效计算加性GPs的后验均值、后验方差、对数似然以及这三个函数的梯度。我们将算法应用于贝叶斯优化，提出了贝叶斯优化中后验更新、超参数学习以及采集函数及其梯度计算的高效算法。给定后验，对于一般学习率，我们的算法将采集函数及其梯度的计算时间复杂度从$O(n^2)$大幅降至$O(\backslash log n)$，对于小学习率甚至可降至$O(1)$。
英文摘要：
Among generalized additive models, additive Matérn Gaussian Processes (GPs) are one of the most popular for scalable high-dimensional problems. Thanks to their additive structure and stochastic differential equation representation, back-fitting-based algorithms can reduce the time complexity of computing the posterior mean from $O(n^3)$ to $O(n\log n)$ time where $n$ is the data size. However, generalizing these algorithms to efficiently compute the posterior variance and maximum log-likelihood remains an open problem. In this study, we demonstrate that for Additive Matérn GPs, not only the posterior mean, but also the posterior variance, log-likelihood, and gradient of these three functions can be represented by formulas involving only sparse matrices and sparse vectors. We show how to use these sparse formulas to generalize back-fitting-based algorithms to efficiently compute the posterior mean, posterior variance, log-likelihood, and gradient of these three functions for additive GPs, all in $O(n \log n)$ time. We apply our algorithms to Bayesian optimization and propose efficient algorithms for posterior updates, hyperparameters learning, and computations of the acquisition function and its gradient in Bayesian optimization. Given the posterior, our algorithms significantly reduce the time complexity of computing the acquisition function and its gradient from $O(n^2)$ to $O(\log n)$ for general learning rate, and even to $O(1)$ for small learning rate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
