arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1509.04063math.NAcs.NA
收藏

迭代加权最小二乘方法的共轭梯度加速

Conjugate gradient acceleration of iteratively re-weighted least squares methods

Technische Universität München(慕尼黑工业大学);RWTH Aachen University(亚琛工业大学)

Massimo Fornasier, Steffen Peter, Holger Rauhut, Stephan Worm

AI总结:

本文研究了通过共轭梯度法加速迭代加权最小二乘方法在压缩感知中的应用,分析了求解二次子问题的精度对收敛性的影响,并展示了该方法在高维情况下的优越性。

摘要:

迭代加权最小二乘(IRLS)是一种用于求解涉及非二次成本函数(可能非凸、非光滑)的最小化问题的方法,这些函数可以描述为一系列二次函数的下确界。这种转换建议了一种算法方案,即通过数值线性代数工具高效求解一系列二次问题。IRLS方法的广泛应用和简单实现使其成为一种多功能算法。然而,尽管其简单性、通用性和优雅的分析,IRLS的复杂性强烈依赖于如何解决连续二次优化问题的解决方案。在重要的特殊情况中,即压缩感知和稀疏恢复问题中,我们理论和数值地研究了如何准确地通过共轭梯度(CG)方法在每次迭代中求解二次问题,以保证收敛。使用CG方法可以显著加快二次子问题的数值求解,特别是当涉及矩阵的快速矩阵-向量乘法(例如利用FFT)时。此外,我们还研究了收敛速率。我们的改进IRLS方法在许多情况下优于现有的第一类方法,如迭代硬阈值(IHT)或快速迭代软阈值算法(FISTA),尤其是在高维情况下。此外,IRLS通常能够从比IHT和FISTA所需的更少测量中恢复稀疏向量。

英文摘要:

Iteratively Re-weighted Least Squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic functions. This transformation suggests an algorithmic scheme that solves a sequence of quadratic problems to be tackled efficiently by tools of numerical linear algebra. Its general scope and its usually simple implementation, transforming the initial non-convex and non-smooth minimization problem into a more familiar and easily solvable quadratic optimization problem, make it a versatile algorithm. However, despite its simplicity, versatility, and elegant analysis, the complexity of IRLS strongly depends on the way the solution of the successive quadratic optimizations is addressed. For the important special case of $\textit{compressed sensing}$ and sparse recovery problems in signal processing, we investigate theoretically and numerically how accurately one needs to solve the quadratic problems by means of the $\textit{conjugate gradient}$ (CG) method in each iteration in order to guarantee convergence. The use of the CG method may significantly speed-up the numerical solution of the quadratic subproblems, in particular, when fast matrix-vector multiplication (exploiting for instance the FFT) is available for the matrix involved. In addition, we study convergence rates. Our modified IRLS method outperforms state of the art first order methods such as Iterative Hard Thresholding (IHT) or Fast Iterative Soft-Thresholding Algorithm (FISTA) in many situations, especially in large dimensions. Moreover, IRLS is often able to recover sparse vectors from fewer measurements than required for IHT and FISTA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑