LSEMINK：一种用于对数-和-指数最小化的修正 Newton-Krylov 方法
LSEMINK: A Modified Newton-Krylov Method for Log-Sum-Exp Minimization
AI总结：
本文提出修正 Newton-Krylov 算法 LSEMINK，通过在线性模型行空间添加位移修正 Hessian，解决对数-和-指数最小化中二次近似无下界的问题，在图像分类和几何规划中显著提升求解速度与可扩展性。
摘要：
本文介绍了 LSEMINK，一种有效的修正 Newton-Krylov 算法，旨在最小化线性模型的对数-和-指数函数。这类问题常见于几何规划和多项逻辑回归等场景。尽管对数-和-指数函数是光滑且凸的，但标准线搜索 Newton 型方法可能效率低下，因为目标函数的二次近似可能从下方无界。为避免这一问题，LSEMINK 通过在线性模型的行空间中添加一个位移来修正 Hessian。我们证明该位移使二次近似从下方有界，并且整体方案在温和假设下收敛到全局极小值。我们的收敛性证明还表明所有迭代点都位于线性模型的行空间中，这在模型参数缺乏直观含义时（如机器学习中常见的情况）具有吸引力。由于 LSEMINK 使用 Krylov 子空间方法计算搜索方向，它只需要与线性模型进行矩阵-向量乘积，这对大规模问题至关重要。我们在图像分类和几何规划上的数值实验表明，与几何规划和自然梯度下降方法相比，LSEMINK 显著减少了求解时间并提高了可扩展性。它的初始收敛速度明显快于标准 Newton-Krylov 方法，这在机器学习等应用中尤其有吸引力。此外，LSEMINK 对由问题的非光滑性引起的病态条件更加鲁棒。我们在 https://github.com/KelvinKan/LSEMINK 上分享了 MATLAB 实现。
英文摘要：
This paper introduces LSEMINK, an effective modified Newton-Krylov algorithm geared toward minimizing the log-sum-exp function for a linear model. Problems of this kind arise commonly, for example, in geometric programming and multinomial logistic regression. Although the log-sum-exp function is smooth and convex, standard line search Newton-type methods can become inefficient because the quadratic approximation of the objective function can be unbounded from below. To circumvent this, LSEMINK modifies the Hessian by adding a shift in the row space of the linear model. We show that the shift renders the quadratic approximation to be bounded from below and that the overall scheme converges to a global minimizer under mild assumptions. Our convergence proof also shows that all iterates are in the row space of the linear model, which can be attractive when the model parameters do not have an intuitive meaning, as is common in machine learning. Since LSEMINK uses a Krylov subspace method to compute the search direction, it only requires matrix-vector products with the linear model, which is critical for large-scale problems. Our numerical experiments on image classification and geometric programming illustrate that LSEMINK considerably reduces the time-to-solution and increases the scalability compared to geometric programming and natural gradient descent approaches. It has significantly faster initial convergence than standard Newton-Krylov methods, which is particularly attractive in applications like machine learning. In addition, LSEMINK is more robust to ill-conditioning arising from the nonsmoothness of the problem. We share our MATLAB implementation at https://github.com/KelvinKan/LSEMINK.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
