基于随机雅可比估计的大规模机器学习的非线性最小二乘法
Nonlinear Least Squares for Large-Scale Machine Learning using Stochastic Jacobian Estimates
摘要:
对于机器学习中的大型非线性最小二乘损失函数,我们利用模型参数的数量通常超过一批数据的属性。这意味着损失的Hessian中的低秩结构,这使得能够有效地计算搜索方向。利用这一特性,我们开发了两种算法,估计雅可比矩阵,并表现良好,相比国家的最先进的方法。
英文摘要:
For large nonlinear least squares loss functions in machine learning we exploit the property that the number of model parameters typically exceeds the data in one batch. This implies a low-rank structure in the Hessian of the loss, which enables effective means to compute search directions. Using this property, we develop two algorithms that estimate Jacobian matrices and perform well when compared to state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。