piCholesky:多项式插值多重Cholesky因子以实现高效的近似交叉验证
piCholesky: Polynomial Interpolation of Multiple Cholesky Factors for Efficient Approximate Cross-Validation
AI总结:
通过多项式插值多个正则化参数下的Hessian矩阵Cholesky因子,实现高效近似交叉验证,减少计算成本并提供误差界分析。
摘要:
在使用牛顿法求解最小二乘问题时,Hessian矩阵的因子化通常是主要成本,尤其在多个正则化参数(λ)值下。我们提出了一种高效方法,通过插值少量λ值下的Hessian矩阵Cholesky因子,从而在仅付出少量成本的情况下,实现最优的hold-out误差最小化。我们为我们的近似方案提供了正式的误差界,并解决了关键的实现挑战,以充分利用现代架构的计算能力。我们对多个数据集进行了详尽的实证分析,以展示我们方法的有效性。
英文摘要:
The dominant cost in solving least-square problems using Newton's method is often that of factorizing the Hessian matrix over multiple values of the regularization parameter ($λ$). We propose an efficient way to interpolate the Cholesky factors of the Hessian matrix computed over a small set of $λ$ values. This approximation enables us to optimally minimize the hold-out error while incurring only a fraction of the cost compared to exact cross-validation. We provide a formal error bound for our approximation scheme and present solutions to a set of key implementation challenges that allow our approach to maximally exploit the compute power of modern architectures. We present a thorough empirical analysis over multiple datasets to show the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。