无约束凸优化的分段保守方法
A piecewise conservative method for unconstrained convex optimization
摘要：
我们考虑一种基于动力系统的连续时间最优化方法，其中一个静止的大质量粒子在由目标函数产生的保守力场中移动，没有任何摩擦。我们提出了一个基于动能平均耗散的重新开始准则，并证明了强凸函数的一个整体收敛结果。利用辛欧拉离散格式，我们得到了一个迭代优化算法。我们已经考虑了一个离散的平均耗散重启计划，但我们也引入了一个新的重启程序的基础上，确保在每次迭代的目标函数的减少大于一个步骤的经典梯度法。对于离散保守算法，这最后一个重新启动标准是能够保证收敛结果。我们将同样的重新启动方案应用于Nesterov加速梯度（NAG-C），并将此重新启动的NAG-C作为数值实验的基准。在光滑凸问题中，我们的方法显示出比重新启动的NAG-C更快的收敛速度。我们提出了一个扩展的离散保守算法的复合优化：在数值测试涉及非强凸函数与$\ell^1$ -正则化，它有更好的性能比著名的高效快速迭代收缩-保持算法，自适应重启计划加速。
英文摘要：
We consider a continuous-time optimization method based on a dynamical system, where a massive particle starting at rest moves in the conservative force field generated by the objective function, without any kind of friction. We formulate a restart criterion based on the mean dissipation of the kinetic energy, and we prove a global convergence result for strongly-convex functions. Using the Symplectic Euler discretization scheme, we obtain an iterative optimization algorithm. We have considered a discrete mean dissipation restart scheme, but we have also introduced a new restart procedure based on ensuring at each iteration a decrease of the objective function greater than the one achieved by a step of the classical gradient method. For the discrete conservative algorithm, this last restart criterion is capable of guaranteeing a convergence result. We apply the same restart scheme to the Nesterov Accelerated Gradient (NAG-C), and we use this restarted NAG-C as benchmark in the numerical experiments. In the smooth convex problems considered, our method shows a faster convergence rate than the restarted NAG-C. We propose an extension of our discrete conservative algorithm to composite optimization: in the numerical tests involving non-strongly convex functions with $\ell^1$-regularization, it has better performances than the well known efficient Fast Iterative Shrinkage-Thresholding Algorithm, accelerated with an adaptive restart scheme.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
