超越客户端平均：随机SCAFFOLD中的客户端无关二阶稳态偏差分量

中文摘要 AI 辅助

现有对随机SCAFFOLD（随机控制平均法）的常步长分析确定了主导的O(γ/N)稳态均值偏差，并表明当客户端数量增加时高阶偏差会持续存在，但未在系数层面识别出首个客户端无关的贡献项。对于全参与的随机SCAFFOLD，在一维同质客户端、固定本地步数H以及有界加性梯度噪声的条件下，我们对N≥2的所有情况统一证明：E_{π_{γ,N,H}}[x] - x⋆ = -[f'''(x⋆)σ²/(4f''(x⋆)²)]·(γ/N) - [f'''(x⋆)σ²/(12f''(x⋆))]·[(H-1)(5H-1)/H]γ² + O_H(γ²/N + γ³)。因此，客户端平均可抑制主导的O(γ/N)偏差，但当该偏差系数非零时，无法消除客户端无关的O(γ²)分量。其机制是间接的：尽管直接控制贡献在全局线性平均中逐路径抵消，但控制项仍会改变轮内本地轨迹及其二阶矩；新的梯度噪声与持续的控制波动因此产生本地二阶矩修正，非二次曲率将其转化为稳态均值偏差。该系数在二次目标函数中会消失。数值实验与预测的系数、其随客户端数量增加的持续性及所述联合余项一致。该结果仅适用于一维同质固定H的设置。

英文摘要

Existing constant-step analysis of stochastic \Scaf{} identifies a leading $O(γ/N)$ stationary mean bias and shows that higher-order bias can persist as the client count increases, but does not identify the first client-independent contribution at coefficient level. For full-participation stochastic \Scaf{} with one-dimensional homogeneous clients, fixed local-step count $H$, and bounded additive gradient noise, we prove, uniformly over $N\ge2$, $$ \begin{aligned} \mathbb{E}_{π_{γ,N,H}}[x]-x^\star ={}& -\frac{f'''(x^\star)σ^2}{4f''(x^\star)^2}\fracγ{N}\\ &- \frac{f'''(x^\star)σ^2}{12f''(x^\star)} \frac{(H-1)(5H-1)}{H}γ^2 +O_H\!\left(\frac{γ^2}{N}+γ^3\right). \end{aligned} $$ Hence client averaging suppresses the leading $O(γ/N)$ bias but does not remove the client-independent $O(γ^2)$ component when its coefficient is nonzero. The mechanism is indirect: although the direct control contribution cancels pathwise in the linear global average, the controls still alter within-round local trajectories and their second moments. Fresh gradient noise and persistent control fluctuations therefore generate local second-moment corrections that nonquadratic curvature converts into stationary mean bias. The coefficient vanishes for quadratic objectives. Numerical experiments are consistent with the predicted coefficient, its persistence as client count increases, and the stated joint remainder. The result is restricted to the one-dimensional homogeneous fixed-$H$ setting.

发表机构

• National Chung Hsing University（国立中兴大学）

机构由 AI 辅助整理，请以论文原文为准。