共享内存与分布式内存中的异步Jacobi和随机Gauss--Seidel方法：统一收敛速率分析
Asynchronous Jacobi and randomized Gauss--Seidel methods in shared and distributed memory: A unified convergence-rate analysis
查看机构详情
- Charles University（查理大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文提出统一异步模型，涵盖共享与分布式内存，分析异步Jacobi和随机Gauss-Seidel方法，在稳定性条件下建立线性收敛，并揭示延迟影响，为异步计算提供理论框架。
中文摘要 AI 辅助
异步迭代方法因减少同步和通信开销，在大规模并行计算中颇具吸引力。然而，现有的收敛速率分析主要集中于共享内存实现，而分布式内存系统引入了更普遍且可能不一致的通信延迟。在本工作中，我们从统一视角重新审视对称正定线性系统的异步Jacobi和随机Gauss--Seidel（RGS）方法。首先，我们引入一个通用异步模型，该模型涵盖共享内存和分布式内存设置，并通过一个共同的坐标更新框架表达这两种方法。然后，我们在显式稳定性条件下建立期望下的线性收敛性。所得的收敛界以代数方式依赖于延迟，通过量$\sqrt{\rho\tau}+\rho\tau$体现，其中$\tau$是最大通信延迟，$\rho$反映底层并行实现的通信模式。特别地，在适当的缩放机制下，当$\rho\tau=O(1)$时，保证的每迭代收敛速率与同步RGS具有相同的渐近阶。这些结果为量化异步性对共享和分布式内存环境中异步Jacobi/RGS方法的影响提供了统一框架。
英文摘要
Asynchronous iterative methods are attractive for large-scale parallel computing because they reduce synchronization and communication overhead. Existing convergence-rate analyses, however, have primarily focused on shared memory implementations, whereas distributed memory systems introduce more general and potentially inconsistent communication delays. In this work, we revisit asynchronous Jacobi and randomized Gauss--Seidel (RGS) methods for symmetric positive definite linear systems from a unified perspective. We first introduce a general asynchronous model that encompasses both shared memory and distributed memory settings and expresses the two methods through a common coordinate-update framework. We then establish linear convergence in expectation under an explicit stability condition. The resulting convergence bound depends algebraically on the delay through the quantity $\sqrt{ρτ}+ρτ$, where $τ$ is the maximum communication delay and $ρ$ reflects the communication pattern of the underlying parallel implementation. In particular, under an appropriate scaling regime with $ρτ=O(1)$, the guaranteed per-iteration convergence rate has the same asymptotic order as that of synchronous RGS. These results provide a unified framework for quantifying the effect of asynchronicity on asynchronous Jacobi/RGS methods in both shared and distributed memory environments.
