随机惯性Krasnosel'skii-Mann迭代实现近最优样本复杂度
Stochastic Inertial Krasnosel'skii-Mann Iteration Achieves Near-Optimal Sample Complexity
- Carnegie Mellon University（卡内基梅隆大学）
- Meta FAIR
- Yale University（耶鲁大学）
- Boston University（波士顿大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出随机惯性Krasnosel'skii-Mann（iKM）方法，通过两次惯性外推实现非扩张算子不动点求解，达到近最优样本复杂度并改进经典KM的收敛速率。
AI中文摘要：
我们分析了一种简单的随机惯性Krasnosel'skii-Mann（iKM）方法，用于在实希尔伯特空间中寻找非扩张算子的不动点。我们的方法仅通过在随机KM [Bravo和Cominetti, 2024]中添加两次惯性外推而获得，每次更新仅调用一次可能具有偏差的随机预言机，并在随机和确定性情形下均达到尖锐的收敛速率。具体而言，在我们提出的参数调度下，我们证明了如下最后迭代不动点残差界：\\[ {O}\left(\frac{1}{K} +\frac{\sigma\log K}{\sqrt K} +\frac{B_K\log K}{K}\right), \\] 其中$K$为时间范围，$\sigma$为噪声水平，$B_K$为累积均方根偏差。当$B_K=O(\sqrt K)$时，这产生了$\widetilde O(\epsilon^{-2})$的样本复杂度，与我们的模型无偏子类下的随机预言机下界[Foster等人, 2019, 定理2]匹配（至多相差对数因子）。它还改进了随机KM [Bravo和Cominetti, 2024, 推论5.4]已知最佳的$O(\epsilon^{-4})$随机迭代保证。据我们所知，这是第一个针对一般非扩张不动点问题的单循环方法，在不使用方差缩减或批处理的情况下达到这一近最优样本复杂度。当预言机精确时，同一方法达到最坏情况最优的$O(K^{-1})$最后迭代残差速率[Park和Ryu, 2022, 定理4.6]，改进了经典KM的$O(K^{-1/2})$速率[Cominetti等人, 2014; Bravo和Cominetti, 2018]。
英文摘要：
We analyze a simple stochastic inertial Krasnosel'skii--Mann (iKM) method for finding a fixed point of a nonexpansive operator in a real Hilbert space. Our method is obtained simply by adding two inertial extrapolations to stochastic KM [Bravo and Cominetti, 2024], and it retains one call to a possibly biased stochastic oracle per update and achieves sharp rates in both the stochastic and deterministic regimes. Specifically, with our proposed parameter schedule, we prove the following last-iterate fixed-point residual bound: \[ {O}\!\left(\frac{1}{K} +\frac{σ\log K}{\sqrt K} +\frac{B_K\log K}{K}\right), \] where $K$ is the horizon, $σ$ is the noise level and $B_K$ is the accumulated root-mean-square bias. When $B_K=O(\sqrt K)$, this yields $\widetilde O(ε^{-2})$ sample complexity that matches, up to a logarithmic factor, the stochastic-oracle lower bound given under the unbiased subclass of our model [Foster et al., 2019, Theorem 2]. It also improves the best-known $O(ε^{-4})$ random-iterate guarantee for stochastic KM [Bravo and Cominetti, 2024, Corollary 5.4]. To our knowledge, this is the first single-loop method for general nonexpansive fixed-point problems to attain this near-optimal sample complexity without variance reduction or batching. When the oracle is exact, the same method attains the worst-case-optimal $O(K^{-1})$ last-iterate residual rate [Park and Ryu, 2022, Theorem 4.6], improving the $O(K^{-1/2})$ rate of classical KM [Cominetti et al., 2014; Bravo and Cominetti, 2018].
