arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.03199math.OCcs.LG
收藏

基于随机线搜索的过参数化模型优化的收敛条件

Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models

Department of Information Engineering, University of Florence(信息工程系,佛罗伦萨大学)

Matteo Lapucci, Davide Pucci

AI总结:

本文针对满足插值条件的过参数化模型有限和拟合问题,研究基于随机线搜索的通用搜索方向优化算法,给出保证回溯有限终止的方向条件及PL函数下线性收敛的附加条件,为相关算法设计提供保障条件参考。

摘要:

本文研究求解与拟合过参数化模型相关的有限和问题的算法,这类模型通常满足插值条件。我们特别关注基于随机线搜索、采用通用搜索方向的方法。我们定义了搜索方向序列的条件,这些条件可保证回溯过程的有限终止性及界的存在。此外,我们阐明了当这类通用算法应用于插值情形下的PL(Polyak-Łojasiewicz)函数时,证明其快速(线性)收敛所需的方向附加性质。从算法设计角度看,所提出的分析确定了可用于相关算法框架的保障条件。具体而言,将随机线搜索集成到动量法、共轭梯度法或自适应预处理方法中可能具有研究价值。

英文摘要:

In this paper, we deal with algorithms to solve the finite-sum problems related to fitting over-parametrized models, that typically satisfy the interpolation condition. In particular, we focus on approaches based on stochastic line searches and employing general search directions. We define conditions on the sequence of search directions that guarantee finite termination and bounds for the backtracking procedure. Moreover, we shed light on the additional property of directions needed to prove fast (linear) convergence of the general class of algorithms when applied to PL functions in the interpolation regime. From the point of view of algorithms design, the proposed analysis identifies safeguarding conditions that could be employed in relevant algorithmic frameworks. In particular, it could be of interest to integrate stochastic line searches within momentum, conjugate gradient or adaptive preconditioning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑