默认稀疏性：高斯过程回归中ARD用于变量选择的理论与实践

中文摘要 AI 辅助

自动相关性确定（ARD）是高斯过程（GP）回归中输入选择的标准工具。通过为协方差核的每个输入赋予单独的长度尺度，并通过最大化边际似然来学习这些长度尺度，ARD让数据决定哪些坐标重要：不相关的输入会获得非常大的长度尺度，从而被有效关闭。我们将这一机制追溯到边际似然所体现的贝叶斯奥卡姆剃刀原理，推导出它剪除输入所依据的梯度，并强调ARD提供的是有效稀疏性而非精确稀疏性。我们回顾了实践中使用的算法以及将长度尺度转化为选择的规则，并综述了渐近理论，区分了长度尺度上的固定域可辨识性障碍与近期为层次高斯过程先验建立的高维选择一致性保证，同时指出普通ARD仍有哪些开放问题。我们将ARD与尖峰-平板先验、稀疏轴对齐和全局-局部收缩先验（包括贝叶斯lasso和马蹄铁先验）、惩罚似然克里金法、敏感性和投影准则以及加性核进行比较。我们认为ARD之所以持久，是因为它与核学习的无缝集成、普遍的软件支持以及低成本，最后我们讨论了其局限性及补救措施。

英文摘要

Automatic relevance determination (ARD) is the standard device for input selection in Gaussian process (GP) regression. By giving the covariance kernel a separate lengthscale for every input and learning those lengthscales by maximizing the marginal likelihood, ARD lets the data decide which coordinates matter: irrelevant inputs receive very large lengthscales and are effectively switched off. We trace this mechanism to the Bayesian Occam's razor embodied in the marginal likelihood, derive the gradient through which it prunes inputs, and emphasize that ARD delivers effective rather than exact sparsity. We review the algorithms used in practice and the rules that turn lengthscales into selections, and we survey the asymptotic theory, distinguishing the fixed-domain identifiability obstruction on the lengthscales from the high-dimensional selection-consistency guarantees recently established for hierarchical GP priors, and noting what remains open for plain ARD. We compare ARD with spike-and-slab priors, sparse axis-aligned and global-local shrinkage priors including the Bayesian lasso and horseshoe, penalized likelihood kriging, sensitivity and projection criteria, and additive kernels. We argue that ARD endures because of its seamless integration with kernel learning, universal software support, and low cost, and we close with its limitations and remedies.

发表机构

• George Mason University（乔治梅森大学）

机构由 AI 辅助整理，请以论文原文为准。