arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.00701math.OC
收藏

快速凸优化:通过梯度动力学的闭环时间缩放

Fast convex optimization via closed-loop time scaling of gradient dynamics

Hedy Attouch, Radu Ioan Bot, Dang-Khoa Nguyen

AI总结:

本文在希尔伯特空间中提出一种基于闭环时间缩放的通用框架,用于设计自适应加速梯度方法,通过反馈控制阻尼实现快速收敛,并扩展到近端算法,数值实验验证了其效率。

摘要:

在希尔伯特空间中,针对凸可微优化问题,我们开发了一个用于自适应加速梯度方法的通用框架。这些方法基于阻尼惯性动力学,其中系数以闭环方式设计。具体而言,阻尼是速度或目标函数梯度的反馈控制。为此,我们开发了作者先前引入的时间缩放和平均技术的闭环版本。由此,我们获得了包含消失粘性阻尼和隐式Hessian驱动阻尼的自主惯性动力学。通过简单地使用连续最速下降法的收敛速率和Jensen不等式,无需进一步的Lyapunov分析,我们证明了轨迹同时具有几个显著性质:它们确保值的快速收敛、梯度向零的快速收敛,并且收敛到最优解。我们的方法导致了并行的算法结果,我们在近端算法的情况下进行了研究。这些是使用自主动力学获得的此类通用结果中最早的一批。由于所提出的数值方法基于近端技术,结果可以扩展到更广泛的类别,特别是最小化适当、下半连续且凸的函数的问题。进行了数值实验以证明所提出方法的效率。

英文摘要:

In a Hilbert setting, for convex differentiable optimization, we develop a general framework for adaptive accelerated gradient methods. They are based on damped inertial dynamics where the coefficients are designed in a closed-loop way. Specifically, the damping is a feedback control of the velocity, or of the gradient of the objective function. For this, we develop a closed-loop version of the time scaling and averaging technique introduced by the authors. We thus obtain autonomous inertial dynamics which involve vanishing viscous damping and implicit Hessian driven damping. By simply using the convergence rates for the continuous steepest descent and Jensen's inequality, without the need for further Lyapunov analysis, we show that the trajectories have several remarkable properties at once: they ensure fast convergence of values, fast convergence of the gradients towards zero, and they converge to optimal solutions. Our approach leads to parallel algorithmic results, that we study in the case of proximal algorithms. These are among the very first general results of this type obtained using autonomous dynamics. Since the proposed numerical methods are based on proximal techniques, the results can be extended to a broader class, specifically to the problem of minimizing a proper, lower semicontinuous, and convex function. Numerical experiments are conducted to demonstrate the efficiency of the proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑