自适应梯度下降中保护步长区间的扩展
Extension of the safeguarding stepsize interval in Adaptive Gradient Descent
浏览论文内容
中文总结 AI 辅助
受AdGD思想启发，提出自适应提供步长区间的方法，通过投影BB步长确保全局收敛并可能加速。还扩大AdGD步长区间保证全局收敛，使能更频繁采用BB步长，最后报告了相关数值实验结果。
中文摘要 AI 辅助
本文受Malitsky和Mishchenko的自适应梯度下降（AdGD）思想启发，提出一种自适应提供步长区间的方法，该区间代表步长确保全局收敛的条件。通过将Barzilai Borwein步长投影到该区间，可确保全局收敛并有望进一步加速收敛。此外，提出扩大AdGD得到的步长区间，并证明在此扩大区间内可保证全局收敛。这种区间扩展使我们能更频繁地采用完整的Barzilai - Borwein（BB）步长。我们报告了将所提区间与BB步长相结合的步长数值实验结果。
英文摘要
In this paper, inspired by the idea of Adaptive Gradient Descent (AdGD) by Malitsky and Mishchenko, we propose a method that adaptively provides a stepsize interval, which represents the condition for stepsizes ensuring global convergence. By projecting the Barzilai Borwein stepsize onto this interval, we can ensure global convergence and expect further acceleration of convergence. Furthermore, we propose enlarging the stepsize interval obtained by AdGD and show that global convergence can be guaranteed within this enlarged stepsize interval. This extension of the interval enables us to adopt the full Barzilai-Borwein (BB) stepsize more frequently. We report the results of numerical experiments on the stepsize that combines the proposed interval with the BB stepsize.
