The ``fast iterative shrinkage-thresholding algorithm'', a.k.a. FISTA, is one of the most widely used algorithms in the literature. However, despite its optimal theoretical $O(1/k^2)$ convergence rate guarantee, oftentimes in practice its performance is not as desired owing to the (local) oscillatory behaviour. Over the years, various approaches are proposed to overcome this drawback of FISTA, in this paper, we propose a simple yet effective modification to the algorithm which has two advantages: 1) it enables us to prove the convergence of the generated sequence; 2) it shows superior practical performance compared to the original FISTA. Numerical experiments are presented to illustrate the superior performance of the proposed algorithm.
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学)
在本文中,我们重新审视类迭代收缩阈值算法(ISTA)解决线性逆问题的稀疏表示,这出现在信号和图像处理。对图像去模糊的数值实验表明,该算法在坐标尺度上的收敛行为是线性的,而不是对数的,近似于平坦的。通过细致的观察,我们发现前面的假设是光滑的部分是凸的,削弱了最小二乘模型。具体地说,假设平滑部分是强凸的对于最小二乘模型是更合理的,即使图像矩阵可能是病态的。此外,我们改进了[Li et al.,2022年]。基于这个关键不等式,我们将复合优化问题的线性收敛推广到目标值和平方次梯度范数下。同时,我们设置了一个简单的病态矩阵来代替原来的模糊矩阵,便于计算奇异值。新的数值实验表明,Nesterov的加速梯度下降(NAG)的强凸函数的近端推广比ISTA具有更快的线性收敛速率。基于更紧的枢轴不等式,我们还推广了更快的线性收敛速率的复合优化,在目标值和平方近次梯度范数,利用构造良好的李雅普诺夫函数与轻微的修改和相空间表示的基础上的高分辨率微分方程框架的隐速度计划。
Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);School of Mathematical Sciences, University of Chinese Academy of Sciences(中国科学院大学数学科学学院)
Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems
School of Electrical Engineering, Tel Aviv University(特拉维夫大学电子工程学院);Electrical Engineering Department, Technion - IIT(技术学院-理工学院电子工程系);Computer Science Department, Technion - IIT(技术学院-理工学院计算机科学系);Electrical and Computer Engineering Department, Duke University(杜克大学电气与计算机工程系)