arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.07805math.OCcs.SYeess.SY
收藏

通过混合控制视角实现连续时间加速方法

Continuous-Time Accelerated Methods via a Hybrid Control Lens

Delft Center for Systems and Control, TU Delft(代尔夫特系统与控制中心,代尔夫特理工大学)

Arman Sharifi Kolarijani, Peyman Mohajerin Esfahani, Tamás Keviczky

AI总结:

本文提出两种基于混合控制系统的快速优化方法,通过动态合成反馈控制实现预设的指数收敛速率,同时满足Polyak-Łojasiewicz不等式条件,确保无局部极值并具有梯度支配性质。

摘要:

将优化方法视为动态系统可以追溯到几个世纪前,以理解优化方法的概念和行为。近年来,这种观点成为设计新优化方法的驱动力。受Nesterov快速方法的最新动态系统观点启发,我们提出两类快速方法,以混合控制系统的形式进行公式化,以获得预设的指数收敛速率。与现有快速方法不同,这些方法不是参数随时间变化的二阶微分方程,而是动态地以状态依赖的方式合成反馈控制。在第一类中,阻尼项被视为控制输入,而在第二类中,梯度影响动态的幅度作为控制器。目标函数需要满足所谓的Polyak--Łojasiewicz不等式,这有效地意味着没有局部极值和一定的梯度支配性质。此外,我们证明这两种混合结构具有无Zeno解轨迹。最后,我们提供了一种机制来确定离散化步长以实现指数收敛速率。

英文摘要:

Treating optimization methods as dynamical systems can be traced back centuries ago in order to comprehend the notions and behaviors of optimization methods. Lately, this mind set has become the driving force to design new optimization methods. Inspired by the recent dynamical system viewpoint of Nesterov's fast method, we propose two classes of fast methods, formulated as hybrid control systems, to obtain pre-specified exponential convergence rate. Alternative to the existing fast methods which are parametric-in-time second order differential equations, we dynamically synthesize feedback controls in a state-dependent manner. Namely, in the first class the damping term is viewed as the control input, while in the second class the amplitude with which the gradient of the objective function impacts the dynamics serves as the controller. The objective function requires to satisfy the so-called Polyak--Łojasiewicz inequality which effectively implies no local optima and a certain gradient-domination property. Moreover, we establish that both hybrid structures possess Zeno-free solution trajectories. We finally provide a mechanism to determine the discretization step size to attain an exponential convergence rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑