一种抽象李雅普诺夫控制优化器：局部镇定与全局收敛
An Abstract Lyapunov Control Optimizer: Local Stabilization and Global Convergence
AI总结：
本文将自适应步长优化器视为连续动力系统的离散化，利用连续选择理论证明其在任意超参数下的李雅普诺夫稳定性与收敛性，并给出Momentum/Heavy-Ball和p-GD算法的新结果。
摘要：
近年来，许多机器学习优化器被分析为某些微分方程在步长趋于零时的渐近极限。换言之，这些优化器可被视为应用于连续动力系统的有限差分格式。但文献中的大部分结果都关注恒定步长算法。本文的主要目的是研究自适应步长对应方法的保证。事实上，这种动力学观点可用于设计步长更新规则，即通过选择连续方程的一种离散化方式来保留其最相关的特征。在这项工作中，我们分析了这类自适应优化器，并证明了它们在任意超参数选择下的李雅普诺夫稳定性和收敛性质。据我们所知，本文首次引入了一般拓扑学中的连续选择理论，以克服由于非恒定且非正则的步长策略所带来的一些内在困难。所建立的一般框架为自适应和恒定步长的Momentum/Heavy-Ball算法以及p-GD算法提供了许多新结果。
英文摘要：
Recently, many machine learning optimizers have been analysed considering them as the asymptotic limit of some differential equations when the step size goes to zero. In other words, the optimizers can be seen as a finite difference scheme applied to a continuous dynamical system. But the major part of the results in the literature concerns constant step size algorithms. The main aim of this paper is to investigate the guarantees of the adaptive step size counterpart. In fact, this dynamical point of view can be used to design step size update rules, by choosing a discretization of the continuous equation that preserves its most relevant features. In this work, we analyse this kind of adaptive optimizers and prove their Lyapunov stability and convergence properties for any choice of hyperparameters. At the best of our knowledge, this paper introduces for the first time the use of continuous selection theory from general topology to overcome some of the intrinsic difficulties due to the non constant and non regular step size policies. The general framework developed gives many new results on adaptive and constant step size Momentum/Heavy-Ball and p-GD algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
