具有行列随机权值的有向图的方差缩减随机最优化
Variance reduced stochastic optimization over directed graphs with row and column stochastic weights
摘要：
本文提出了一种一阶分布式随机最优化算法--AB-SAGA，用于极小化分布在任意有向图上的光滑强凸函数的有限和。AB-SAGA使用节点级方差减少来消除由随机梯度引起的不确定性，并随后采用网络级梯度跟踪来解决节点之间的数据不相似性。与现有的方法，使用非线性推和校正，以消除由定向通信所造成的不平衡，在AB-SAGA的共识更新是线性的，并使用行和列的随机权重。我们表明，对于一个恒定的步长，AB-SAGA线性收敛到全局最优。我们量化的基本图形使用一个明确的方向性常数和特征的制度，其中AB-SAGA实现了线性加速超过其集中的对应的有向性。数值实验表明，AB-SAGA的收敛性强凸和非凸问题。
英文摘要：
This paper proposes AB-SAGA, a first-order distributed stochastic optimization method to minimize a finite-sum of smooth and strongly convex functions distributed over an arbitrary directed graph. AB-SAGA removes the uncertainty caused by the stochastic gradients using a node-level variance reduction and subsequently employs network-level gradient tracking to address the data dissimilarity across the nodes. Unlike existing methods that use the nonlinear push-sum correction to cancel the imbalance caused by the directed communication, the consensus updates in AB-SAGA are linear and uses both row and column stochastic weights. We show that for a constant step-size, AB-SAGA converges linearly to the global optimal. We quantify the directed nature of the underlying graph using an explicit directivity constant and characterize the regimes in which AB-SAGA achieves a linear speed-up over its centralized counterpart. Numerical experiments illustrate the convergence of AB-SAGA for strongly convex and nonconvex problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
