快速矩阵平方根及其在高斯过程和贝叶斯优化中的应用
Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization
摘要：
矩阵平方根及其逆经常出现在机器学习中，例如，当从高维高斯$\mathcal{N}(\mathbf 0, \mathbf K)$采样或相对于协方差矩阵$\mathbf K$白化向量$\mathbf b$时。虽然现有的方法通常需要$O(N^3)$计算，我们引入了一个高效的二次时间算法计算$\mathbf K^{1/2} \mathbf b$，$\mathbf K^{-1/2} \mathbf b$，和他们的衍生物通过矩阵向量乘法（MVMs）。我们的方法结合了Krylov子空间方法与有理逼近，通常实现$4$小数位的准确性与少于$100$的MVMs。此外，反向传递需要很少的额外计算。我们证明了我们的方法对像$50,\!000 \times 50,\!000$这样大的矩阵的适用性-远远超出了传统方法-而且逼近误差很小。将这种增强的可扩展性应用于变分高斯过程、贝叶斯优化和吉布斯采样，可以产生更强大的模型，具有更高的准确性。
英文摘要：
Matrix square roots and their inverses arise frequently in machine learning, e.g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$. While existing methods typically require $O(N^3)$ computation, we introduce a highly-efficient quadratic-time algorithm for computing $\mathbf K^{1/2} \mathbf b$, $\mathbf K^{-1/2} \mathbf b$, and their derivatives through matrix-vector multiplication (MVMs). Our method combines Krylov subspace methods with a rational approximation and typically achieves $4$ decimal places of accuracy with fewer than $100$ MVMs. Moreover, the backward pass requires little additional computation. We demonstrate our method's applicability on matrices as large as $50,\!000 \times 50,\!000$ - well beyond traditional methods - with little approximation error. Applying this increased scalability to variational Gaussian processes, Bayesian optimization, and Gibbs sampling results in more powerful models with higher accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
