Yanjun Zhang, Hanyu Li
In this paper, combining count sketch and maximal weighted residual Kaczmarz method, we propose a fast randomized algorithm for large overdetermined linear systems. Convergence analysis of the new algorithm is provided. Numerical experiments show that, for the same accuracy, our method behaves better in computing time compared with the state-of-the-art algorithm.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this paper, an extension of Kaczmarz method, the Kaczmarz method with oblique projection (KO), is introduced and analyzed. Using this method, a number of iteration steps to solve the over-determined systems of linear equations are significantly reduced, and the the computing time is much saved, especially for those problems that contain some linear equations with near-linear correlation. Simultaneously, a randomized version--randomized Kaczmarz method with oblique projection (RKO) is established. The convergence proofs of these two methods are given and numerical experiments show the effect…
Preconditioning Kaczmarz method by sketching
本文提出了一种通过采样预条件化Kaczmarz方法的新方法，利用QR分解构建最优预条件器，但其复杂度过高，因此采用随机采样降低复杂度，并在随机和实际过定线性系统中进行了数值实验。
The block Kaczmarz method is an iterative scheme for solving overdetermined least-squares problems. At each step, the algorithm projects the current iterate onto the solution space of a subset of the constraints. This paper describes a block Kaczmarz algorithm that uses a randomized control scheme to choose the subset at each step. This algorithm is the first block Kaczmarz method with an (expected) linear rate of convergence that can be expressed in terms of the geometric properties of the matrix and its submatrices. The analysis reveals that the algorithm is most effective when it is given a…
Two-subspace Projection Method for Coherent Overdetermined Systems
提出一种基于两个随机行投影到解空间的双子空间Kaczmarz方法，实现期望指数收敛，在行相关系统中优于随机Kaczmarz方法。
Accelerated Kaczmarz methods via randomized sketch techniques for solving consistent linear systems
针对相容线性系统求解问题，本文采用两种随机草图技术改进两类Kaczmarz方法，完成收敛性分析与解偏差上界推导，实验验证新方法在同等精度下运行效率更优。
Two-subspace Projection Method for Coherent Overdetermined Systems (Technical Report)
提出一种基于双子空间投影的POCS算法，通过迭代投影到两个随机行定义的解空间，在相干超定线性系统中实现指数收敛，显著优于随机Kaczmarz方法。
Acceleration of Randomized Kaczmarz Method via the Johnson-Lindenstrauss Lemma
本文提出一种改进的随机Kaczmarz方法，通过从随机选择的集合中选择最优投影，并利用Johnson-Lindenstrauss降维技术保持运行时间，显著加速收敛。
The sketch-and-project (SAP) framework for solving systems of linear equations has unified the theory behind popular projective iterative methods such as randomized Kaczmarz, randomized coordinate descent, and variants thereof. The randomized extended Kaczmarz (REK) method is a popular extension of randomized Kaczmarz for solving inconsistent systems, which has not yet been shown to lie within the SAP framework. In this work we show that, in a certain sense, REK may be expressed as the limit point of a family of SAP methods, but we argue that it is unlikely that REK can be translated into a SA…
A randomized Kaczmarz algorithm with exponential convergence
针对超定相容线性方程组，提出一种随机Kaczmarz算法，证明其期望收敛速度为指数级，且与方程数量无关，在极度过定系统中优于现有方法。
A fast randomized Kaczmarz algorithm for sparse solutions of consistent linear systems
本文提出一种改进的随机Kaczmarz算法，通过加权行投影加速稀疏解求解，适用于一致线性系统。
Accelerated Kaczmarz Algorithms using History Information
本文提出两种方法加速随机Kaczmarz算法，利用历史估计生成预条件器或结合SAG方法提升收敛速度，数值实验显示性能显著优于标准算法。
Accelerating Random Kaczmarz Algorithm Based on Clustering Information
本文提出基于聚类信息的加速随机Kaczmarz算法，利用Johnson-Lindenstrauss引理改进块Kaczmarz和普通Kaczmarz算法，并理论证明了收敛性的提升。
We propose using greedy and randomized Kaczmarz inner-iterations as preconditioners for the right-preconditioned flexible GMRES method to solve consistent linear systems, with a parameter tuning strategy for adjusting the number of inner iterations and the relaxation parameter. We also present theoretical justifications of the right-preconditioned flexible GMRES for solving consistent linear systems. Numerical experiments on overdetermined and underdetermined linear systems show that the proposed method is superior to the GMRES method preconditioned by NE-SOR inner iterations in terms of total…
The sketch-and-project, as a general archetypal algorithm for solving linear systems, unifies a variety of randomized iterative methods such as the randomized Kaczmarz and randomized coordinate descent. However, since it aims to find a least-norm solution from a linear system, the randomized sparse Kaczmarz can not be included. This motivates us to propose a more general framework, called sketched Bregman projection (SBP) method, in which we are able to find solutions with certain structures from linear systems. To generalize the concept of adaptive sampling to the SBP method, we show how the …
The Kaczmarz algorithm is one of the most popular methods for solving large-scale over-determined linear systems due to its simplicity and computational efficiency. This method can be viewed as a special instance of a more general class of sketch and project methods. Recently, a block Gaussian version was proposed that uses a block Gaussian sketch, enjoying the regularization properties of Gaussian sketching, combined with the acceleration of the block variants. Theoretical analysis was only provided for the non-block version of the Gaussian sketch method. Here, we provide theoretical guarante…
Read Between the Hyperplanes: On Spectral Projection and Sampling Approaches to Randomized Kaczmarz
本文提出通过行间关系和谱多样性行采样改进随机Kaczmarz方法，以加速病态和过定线性系统的收敛。
On the Relation Between the Randomized Extended Kaczmarz Algorithm and Coordinate Descent
本文比较了随机扩展Kaczmarz算法与随机坐标下降算法在求解满秩过定线性最小二乘问题中的效率，证明坐标下降在满足相同残差终止条件时操作更少，且先用坐标下降计算残差再用标准Kaczmarz求解一致系统比扩展Kaczmarz更高效。
Randomized iterative algorithms have recently been proposed to solve large-scale linear systems. In this paper, we present a simple randomized extended block Kaczmarz algorithm that exponentially converges in the mean square to the unique minimum $\ell_2$-norm least squares solution of a given linear system of equations. The proposed algorithm is pseudoinverse-free and therefore different from the projection-based randomized double block Kaczmarz algorithm of Needell, Zhao, and Zouzias. We emphasize that our method works for all types of linear systems (consistent or inconsistent, overdetermin…
The standard form and convergence theory of the relaxation Kaczmarz-Tanabe method for solving linear systems
本文针对大规模线性系统求解，将松弛参数引入 Kaczmarz-Tanabe 方法，提出了其标准形式，证明了收敛的充分条件，并通过数值实验验证了收敛特性。
Quantile Randomized Kaczmarz Algorithm with Whitelist Trust Mechanism
分位数随机Kaczmarz算法结合白名单信任机制，通过在线检测器和分位数估计提升超定系统在噪声下的鲁棒性和收敛效率。
Block Kaczmarz Method with Inequalities
本文提出一种块型方法用于混合等式和不等式系统，通过矩阵分块提升收敛性，并证明线性收敛率，同时探讨不等式块在特定几何条件下同样有效。
A multilevel sketch-and-solve method for overdetermined least squares problems
提出一种多级草图与求解（MLSAS）方法，通过结合不同草图大小的样本和校正项，在保持计算效率的同时提高超定最小二乘问题的估计精度。
We develop two fundamental stochastic sketching techniques; Penalty Sketching (PS) and Augmented Lagrangian Sketching (ALS) for solving consistent linear systems. The proposed PS and ALS techniques extend and generalize the scope of Sketch & Project (SP) method by introducing Lagrangian penalty sketches. In doing so, we recover SP methods as special cases and furthermore develop a family of new stochastic iterative methods. By varying sketch parameters in the proposed PS method, we recover novel stochastic methods such as Penalty Newton Descent, Penalty Kaczmarz, Penalty Stochastic Descent, Pe…
Randomized Kaczmarz solver for noisy linear systems
针对带噪声的线性系统Ax≈b+r，提出随机Kaczmarz方法，证明其以与无噪声情况相同的指数收敛速率达到依赖于矩阵A的误差阈值。
Extension of Sparse Randomized Kaczmarz Algorithm for Multiple Measurement Vectors
本文提出改进的随机Kaczmarz算法解决多测量向量问题，通过共同稀疏支持实现更高效的恢复与收敛。
On Convergence Behavior of Randomized Kaczmarz-type Methods for Solving Doubly Noisy Linear Systems
本文针对双重噪声线性系统，首次分析了REK、RBK、RDBK等随机Kaczmarz型算法的收敛性，证明其收敛到无噪声系统最小二乘解邻域，并通过数值实验验证了理论结果。
On greedy multi-step inertial randomized Kaczmarz method for solving linear systems
本文针对大规模线性系统求解问题，提出融合贪心概率准则与更严格阈值的贪心多步惯性随机Kaczmarz方法，证明其收敛性更优，阐明多步惯性外推的几何解释及与两类投影方法的关联，通过数值实验验证结论。
On global randomized block Kaczmarz algorithm for solving large-scale matrix equations
本文提出两种全局随机块Kaczmarz算法，用于高效求解大规模线性矩阵方程，并在图像去模糊中表现出色。
Convergence of the randomized Kaczmarz method for phase retrieval
本文研究了随机Kaczmarz方法在相位恢复问题中的收敛性，证明了在高概率下，随机测量系统能保证均方收敛，收敛速度与线性设置相当。
We develop two greedy sampling rules for the Sketch & Project method for solving linear feasibility problems. The proposed greedy sampling rules generalize the existing max-distance sampling rule and uniform sampling rule and generate faster variants of Sketch & Project methods. We also introduce greedy capped sampling rules that improve the existing capped sampling rules. Moreover, we incorporate the so-called heavy ball momentum technique to the proposed greedy Sketch & Project method. By varying the parameters such as sampling rules, sketching vectors; we recover several well-known algorith…
We provide a complete characterization of the randomized Kaczmarz algorithm (RKA) for inconsistent linear systems. The Kaczmarz algorithm, known in some fields as the algebraic reconstruction technique, is a classical method for solving large-scale overdetermined linear systems through a sequence of projection operators; the randomized Kaczmarz algorithm is a recent proposal by Strohmer and Vershynin to randomize the sequence of projections in order to guarantee exponential convergence (in mean square) to the solutions. A flurry of work followed this development, with renewed interest in the a…
We present new adaptive sampling rules for the sketch-and-project method for solving linear systems. To deduce our new sampling rules, we first show how the progress of one step of the sketch-and-project method depends directly on a sketched residual. Based on this insight, we derive a 1) max-distance sampling rule, by sampling the sketch with the largest sketched residual 2) a proportional sampling rule, by sampling proportional to the sketched residual, and finally 3) a capped sampling rule. The capped sampling rule is a generalization of the recently introduced adaptive sampling rules for t…
A sketch-and-project method for solving the matrix equation AXB = C
提出一种基于优化问题的草图与投影方法求解矩阵方程 AXB=C，给出收敛速率下界与含秩亏缺情形的收敛条件，并通过参数选择恢复多种已知算法及 GaussGRK，数值实验验证其有效性。
Fast and forward stable randomized algorithms for linear least-squares problems
针对超定线性最小二乘问题，本文证明合理实现的迭代草图随机算法兼具前向稳定性与高效性，通过数值实验验证其在大规模问题上比基于QR的求解器更快且稳定，解决了此前草图预处理算法存在数值不稳定性的相关疑问。
Linear Convergence of Reshuffling Kaczmarz Methods With Sparse Constraints
本文研究带稀疏约束的 Kaczmarz 型迭代方法，证明重洗 KZIHT 的线性收敛性，并提出周期阈值 KZPT，在 BOS 与次高斯矩阵模型下建立保证且性能更优。
Recently proposed adaptive Sketch & Project (SP) methods connect several well-known projection methods such as Randomized Kaczmarz (RK), Randomized Block Kaczmarz (RBK), Motzkin Relaxation (MR), Randomized Coordinate Descent (RCD), Capped Coordinate Descent (CCD), etc. into one framework for solving linear systems. In this work, we first propose a Stochastic Steepest Descent (SSD) framework that connects SP methods with the well-known Steepest Descent (SD) method for solving positive-definite linear system of equations. We then introduce two greedy sampling strategies in the SSD framework that…
We describe two algorithms to efficiently solve regularized linear least squares systems based on sketching. The algorithms compute preconditioners for $\min \|Ax-b\|^2_2 + λ\|x\|^2_2$, where $A\in\mathbb{R}^{m\times n}$ and $λ>0$ is a regularization parameter, such that LSQR converges in $\mathcal{O}(\log(1/ε))$ iterations for $ε$ accuracy. We focus on the context where the optimal regularization parameter is unknown, and the system must be solved for a number of parameters $λ$. Our algorithms are applicable in both the underdetermined $m\ll n$ and the overdetermined $m\gg n$ setting. Firstly…
PLSS: A Projected Linear Systems Solver
针对相容线性系统，本文提出每次迭代扩展草图矩阵并产生正交更新的 PLSS 迭代投影方法，可有限步收敛并推广 Kaczmarz，在大型稀疏问题上优于或匹敌现有求解器。
Randomized iterative methods for generalized absolute value equations: Solvability and error bounds
本文研究非方系数矩阵广义绝对值方程的随机迭代求解，建立可解性充要条件与误差界，并提出涵盖 Picard 迭代和随机 Kaczmarz 方法的统一算法框架，证明其几乎必然收敛和线性收敛速率。
Fast randomized least-squares solvers can be just as accurate and stable as classical direct solvers
针对现有随机最小二乘求解器无向后稳定性的问题，提出SPIR和FOSSILS两种可证明向后稳定的求解器，结合迭代精化与预条件迭代方法，收敛速度与现有算法相同，有望整合到软件库中替代经典求解器。
On the adaptive deterministic block coordinate descent methods with momentum for solving large linear least-squares problems
本文提出带动量的自适应确定性块坐标下降法 mADBCD，并结合 count sketch 形成 CS-mADBCD，证明满秩下线性收敛，实验显示其在 CPU 时间与迭代数上更优。
Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems
本文提出一种高效的加速坐标下降方法，通过改进Nesterov的方法，提升多种算法的收敛速度，并在不同情况下实现更快的线性系统求解。
Subspace-constrained randomized coordinate descent for linear systems with good low-rank matrix approximations
本文提出了一种子空间约束随机坐标下降法，用于求解具有良好低秩矩阵近似的线性系统，该方法能有效应对大谱异常值问题。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。