基于Flip-Flop洗牌与锚定的随机外梯度方法：可证明的改进
Stochastic Extragradient with Flip-Flop Shuffling & Anchoring: Provable Improvements
AI总结：
本文针对无约束有限和极小极大问题中随机外梯度方法收敛困难的问题，提出结合flip-flop洗牌与锚定技巧的SEG-FFA方法，并证明其在凸-凹及强凸-强凹情形下均收敛且具有更快的收敛速率。
摘要：
在极小极大优化中，外梯度（EG）方法已被广泛研究，因为它在凸-凹（C-C）问题中优于梯度下降-上升方法。然而，随机EG（SEG）在C-C问题中取得的成功有限，尤其是在无约束情形下。受近期基于洗牌的随机方法进展的启发，我们研究了基于洗牌的SEG在无约束有限和极小极大问题中的收敛性，以寻找收敛的基于洗牌的SEG。我们的分析表明，无论是随机重排还是最近提出的flip-flop洗牌，单独使用都可能在C-C问题中出现发散。然而，借助一种称为锚定（anchoring）的额外简单技巧，我们开发了带flip-flop锚定的SEG（SEG-FFA）方法，该方法在C-C问题中成功收敛。我们还在强凸-强凹情形下给出了上界和下界，证明与其他基于洗牌的方法相比，SEG-FFA具有可证明的更快的收敛速率。
英文摘要：
In minimax optimization, the extragradient (EG) method has been extensively studied because it outperforms the gradient descent-ascent method in convex-concave (C-C) problems. Yet, stochastic EG (SEG) has seen limited success in C-C problems, especially for unconstrained cases. Motivated by the recent progress of shuffling-based stochastic methods, we investigate the convergence of shuffling-based SEG in unconstrained finite-sum minimax problems, in search of convergent shuffling-based SEG. Our analysis reveals that both random reshuffling and the recently proposed flip-flop shuffling alone can suffer divergence in C-C problems. However, with an additional simple trick called anchoring, we develop the SEG with flip-flop anchoring (SEG-FFA) method which successfully converges in C-C problems. We also show upper and lower bounds in the strongly-convex-strongly-concave setting, demonstrating that SEG-FFA has a provably faster convergence rate compared to other shuffling-based methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
