发表机构
Vrije Universiteit Brussel; University of Antwerp（布鲁塞尔自由大学; 安特卫普大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对稀疏高斯过程回归，提出三种基于信息论的基函数选择标准，在六个UCI基准上验证了其优于传统截断方法，尤其对HSGP和VISH族有显著提升。
AI 中文摘要
稀疏高斯过程通过在输入空间上的固定基$\{\phi_j\}$中用适当的展开替换核函数，实现了$O(N)$的推理复杂度。给定计算预算$M \ll N$，实践者通常将基截断为前$M$项。然而，形式体系并不阻止仅选择对当前数据重要的$M$个基函数。这样可以避免将预算浪费在没有信号的基函数上，但需要一种对候选基函数进行排序的标准。我们提出了三种基于信息论视角的基函数选择问题的标准。每种标准对应选择时不同的知识状态：无数据状态、无先验状态和介于两者之间的状态。然后，我们在六个UCI回归基准上，跨三个基函数族：希尔伯特空间高斯过程（HSGP）、变分傅里叶特征（VFF）和变分诱导球谐函数（VISH），研究了截断与选择策略的性能。我们观察到，无数据标准是一种安全的默认选择，对于HSGP、VFF和VISH，它匹配或优于截断，对VISH有显著提升，并且优于最近为该基函数族开发的选择启发式方法。数据感知的无先验和介于两者之间的标准，特别是对于HSGP（在实践中三个族中使用最广泛的）相比截断提供了显著增益。
英文摘要
Sparse Gaussian processes achieve $O(N)$ inference by replacing the kernel with an appropriate expansion in a fixed basis $\{ϕ_j\}$ on the input space. Given a compute budget $M \ll N$, practitioners conventionally truncate the basis to its first $M$ entries. Nothing in the formalism, however, prevents one from selecting only those $M$ basis functions that matter for the data at hand. This would avoid spending budget on basis functions where there is no signal, but it requires a criterion for ranking the candidates. We propose three such criteria derived from an information-theoretic view of the basis-function selection problem. Each criterion matches a different state of knowledge at selection time: a no-data state, a no-prior state, and an in-between state. We then study the performance of truncation versus selection strategies on six UCI regression benchmarks across three basis families: Hilbert-space Gaussian processes (HSGP), variational Fourier features (VFF), and variational inducing spherical harmonics (VISH). We observe that the no-data criterion is a safe default, matching or improving on truncation for HSGP, VFF and VISH, with substantial gains for VISH and improvements over a recently developed selection heuristic for that basis family. The data-aware no-prior and in-between criteria provide substantial gains over truncation specifically for HSGP, which is the most broadly used of the three families in practice.
Comments18 pages, 8 figures
