arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.02346cs.LGstat.ML
收藏

利用 Hankel-Toeplitz 结构快速计算核精度矩阵

Exploiting Hankel-Toeplitz Structures for Fast Computation of Kernel Precision Matrices

Delft University of Technology(代尔夫特理工大学);Linköping University(林雪平大学);Aalto University(阿尔托大学)

Frida Viset, Anton Kullberg, Frederiek Wesel, Arno Solin

AI总结:

本文发现 GP 核精度矩阵可分解为 Hankel-Toeplitz 矩阵之和,从而将预计算复杂度由 O(NM^2) 降至 O(NM),并给出适用于 HGP、VFF 等模型的无额外近似加速条件。

摘要:

Hilbert 空间高斯过程(HGP)方法通过将 GP 投影到 M 个基函数上,提供了一种独立于超参数的基函数近似,以加速高斯过程(GP)推断。这些性质使其在超参数优化期间具有有利的、与数据无关的 O(M^3) 计算复杂度,但需要一次占主导地位的精度矩阵预计算,其开销为 O(NM^2) 次运算。本文在不引入额外近似的情况下,将这一占主导地位的计算复杂度降至 O(NM)。之所以能够做到这一点,是因为我们认识到精度矩阵可以分解为若干 Hankel-Toeplitz 矩阵之和,而每个矩阵仅有 O(M) 个唯一元素。基于这一认识,我们建议仅以 O(NM) 的代价计算这些唯一元素。此外,我们提出两个定理,给出了使该复杂度降低能够广泛适用于多种其他近似 GP 模型(如 Variational Fourier Feature,VFF,方法)的充分条件。这两个定理在不假设数据、也不对 GP 模型本身作额外近似的情况下实现这一点。因此,我们的贡献为若干现有且广泛使用的 GP 近似提供了纯粹的加速,而没有进一步近似。

英文摘要:

The Hilbert-space Gaussian Process (HGP) approach offers a hyperparameter-independent basis function approximation for speeding up Gaussian Process (GP) inference by projecting the GP onto M basis functions. These properties result in a favorable data-independent $\mathcal{O}(M^3)$ computational complexity during hyperparameter optimization but require a dominating one-time precomputation of the precision matrix costing $\mathcal{O}(NM^2)$ operations. In this paper, we lower this dominating computational complexity to $\mathcal{O}(NM)$ with no additional approximations. We can do this because we realize that the precision matrix can be split into a sum of Hankel-Toeplitz matrices, each having $\mathcal{O}(M)$ unique entries. Based on this realization we propose computing only these unique entries at $\mathcal{O}(NM)$ costs. Further, we develop two theorems that prescribe sufficient conditions for the complexity reduction to hold generally for a wide range of other approximate GP models, such as the Variational Fourier Feature (VFF) approach. The two theorems do this with no assumptions on the data and no additional approximations of the GP models themselves. Thus, our contribution provides a pure speed-up of several existing, widely used, GP approximations, without further approximations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑