Accelerating the alternating projection algorithm for the case of affine subspaces using supporting hyperplanes

Set intersection problems: Supporting hyperplanes and quadratic programming

We study how the supporting hyperplanes produced by the projection process can complement the method of alternating projections and its variants for the convex set intersection problem. For the problem of finding the closest point in the intersection of closed convex sets, we propose an algorithm that, like Dykstra's algorithm, converges strongly in a Hilbert space. Moreover, this algorithm converges in finitely many iterations when the closed convex sets are cones in $\mathbb{R}^{n}$ satisfying an alignment condition. Next, we propose modifications of the alternating projection algorithm, and…

Infeasibility and error bound imply finite convergence of alternating projections

This paper combines two ingredients in order to get a rather surprising result on one of the most studied, elegant and powerful tools for solving convex feasibility problems, the method of alternating projections (MAP). Going back to names such as Kaczmarz and von Neumann, MAP has the ability to track a pair of points realizing minimum distance between two given closed convex sets. Unfortunately, MAP may suffer from arbitrarily slow convergence, and sublinear rates are essentially only surpassed in the presence of some Lipschitzian error bound, which is our first ingredient. The second one is …

利用外推加速非负矩阵因子分解算法

Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation

本文提出了一种通用框架，用于显著加速非负矩阵因子分解（NMF）算法。该框架灵感来源于用于加速凸优化中梯度方法的外推方案以及平行切线法。然而，在处理非凸NMF问题的双块精确坐标下降算法中使用外推是新颖的。我们通过合成、图像和文档数据集，展示了该方法在两个最先进的NMF算法（加速层次交替最小二乘法（A-HALS）和交替非负最小二乘法（ANLS））上的性能。

Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent

Low-rank matrix estimation is a canonical problem that finds numerous applications in signal processing, machine learning and imaging science. A popular approach in practice is to factorize the matrix into two compact low-rank factors, and then optimize these factors directly via simple iterative methods such as gradient descent and alternating minimization. Despite nonconvexity, recent literatures have shown that these simple heuristics in fact achieve linear convergence when initialized properly for a growing number of problems of interest. However, upon closer examination, existing approach…

The Block-wise Circumcentered-Reflection Method

The elementary Euclidean concept of circumcenter has recently been employed to improve two aspects of the classical Douglas--Rachford method for projecting onto the intersection of affine subspaces. The so-called circumcentered-reflection method is able to both accelerate the average reflection scheme by the Douglas--Rachford method and cope with the intersection of more than two affine subspaces. We now introduce the technique of circumcentering in blocks, which, more than just an option over the basic algorithm of circumcenters, turns out to be an elegant manner of generalizing the method of…

带非支撑性 no-good 最优割的整数 L 形方法

Integer L-Shaped Method with Non-Supporting No-Good Optimality Cuts

本文提出一种改进的整数 L 形方法，通过提前终止混合整数子问题生成非支撑性 no-good 最优割，以降低割生成计算成本，并在工业案例中显著减少求解时间或最优性间隙。