arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.13170stat.MLcs.LG
收藏

变分正交特征

Variational Orthogonal Features

University of Cambridge(剑桥大学);Imperial College London(伦敦帝国理工学院)

David R. Burt, Carl Edward Rasmussen, Mark van der Wilk

摘要:

稀疏随机变分推断允许高斯过程模型应用于大型数据集。用这种方法进行推理的每次迭代计算成本是$\mathcal{O}(\tilde{N}M^2+M^3),$,其中$\tilde{N}$是minibatch中的点的数量,$M$是“诱导特征”的数量,其确定变分族的表达性。最近的几项工作表明,对于某些先验,可以定义的功能,删除$\mathcal{O}(M^3)$成本计算的证据下限(ELBO)的小批量估计。这表示当使用$M\gg \tilde{N}$ .我们提出了一个建设的功能,任何固定的前内核,允许使用$T$蒙特卡罗样本在$\mathcal{O}(\tilde{N}T+M^2T)$和$\mathcal{O}(\tilde{N}T+MT)$与额外的近似计算的ELBO的无偏估计量。我们分析了这种额外的近似推理质量的影响。

英文摘要:

Sparse stochastic variational inference allows Gaussian process models to be applied to large datasets. The per iteration computational cost of inference with this method is $\mathcal{O}(\tilde{N}M^2+M^3),$ where $\tilde{N}$ is the number of points in a minibatch and $M$ is the number of `inducing features', which determine the expressiveness of the variational family. Several recent works have shown that for certain priors, features can be defined that remove the $\mathcal{O}(M^3)$ cost of computing a minibatch estimate of an evidence lower bound (ELBO). This represents a significant computational savings when $M\gg \tilde{N}$. We present a construction of features for any stationary prior kernel that allow for computation of an unbiased estimator to the ELBO using $T$ Monte Carlo samples in $\mathcal{O}(\tilde{N}T+M^2T)$ and in $\mathcal{O}(\tilde{N}T+MT)$ with an additional approximation. We analyze the impact of this additional approximation on inference quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑