arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.01633cs.LG
收藏

使用二叉树核的对数线性时间高斯过程

Log-Linear-Time Gaussian Processes Using Binary Tree Kernels

University of Oxford(牛津大学);Meta

Michael K. Cohen, Samuel Daulton, Michael A. Osborne

AI总结:

提出一种“二叉树”核,将数据点置于二叉树叶子上,仅依赖最深共同祖先深度,实现 $O((n+m)\log(n+m))$ 时间的高斯过程回归,在多数任务中预测精度和速度优于稀疏及Matérn方法。

摘要:

高斯过程(Gaussian processes, GPs)能产生良好的函数概率模型,但大多数 GP 核需要 $O((n+m)n^2)$ 时间,其中 $n$ 是数据点数量,$m$ 是预测位置数量。我们提出了一种新核,允许在 $O((n+m)\log(n+m))$ 时间内进行高斯过程回归。我们的“二叉树”核将所有数据点置于二叉树的叶子上,核仅取决于最深共同祖先的深度。我们可以在 $O(n \log n)$ 时间内将生成的核矩阵作为稀疏秩一矩阵之和存储在 $O(n)$ 空间中,并在 $O(n)$ 时间内近似求逆核矩阵。稀疏 GP 方法也提供线性运行时间,但它们的预测效果不如高维核。在经典的回归任务套件上,我们将我们的核与 Matérn、稀疏和稀疏变分核进行比较。二叉树 GP 在多数数据集上为测试数据分配了最高似然,通常比稀疏方法实现更低的均方误差,并且经常与 Matérn GP 平手或击败它。在大型数据集上,二叉树 GP 最快,比 Matérn GP 快得多。

英文摘要:

Gaussian processes (GPs) produce good probabilistic models of functions, but most GP kernels require $O((n+m)n^2)$ time, where $n$ is the number of data points and $m$ the number of predictive locations. We present a new kernel that allows for Gaussian process regression in $O((n+m)\log(n+m))$ time. Our "binary tree" kernel places all data points on the leaves of a binary tree, with the kernel depending only on the depth of the deepest common ancestor. We can store the resulting kernel matrix in $O(n)$ space in $O(n \log n)$ time, as a sum of sparse rank-one matrices, and approximately invert the kernel matrix in $O(n)$ time. Sparse GP methods also offer linear run time, but they predict less well than higher dimensional kernels. On a classic suite of regression tasks, we compare our kernel against Matérn, sparse, and sparse variational kernels. The binary tree GP assigns the highest likelihood to the test data on a plurality of datasets, usually achieves lower mean squared error than the sparse methods, and often ties or beats the Matérn GP. On large datasets, the binary tree GP is fastest, and much faster than a Matérn GP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑