CED-EF：用于多智能体学习的带误差反馈的压缩精确扩散算法
CED-EF: Compressed Exact Diffusion with Error Feedback for Multi-Agent Learning
浏览论文内容
中文总结 AI 辅助
提出带误差反馈的压缩精确扩散算法CED-EF，解决多智能体网络压缩通信下的分布式随机优化问题，其收敛率与暂态时间特性优于现有结果，数值实验验证了性能优势。
中文摘要 AI 辅助
我们研究N个智能体组成的网络在压缩通信下的分布式随机优化问题，提出带误差反馈的精确扩散算法CED-EF，该算法可直接适配有偏δ-压缩器，且每个节点每次迭代仅传输一个压缩后的模型大小向量。对于方差由σ²（σ≥0）界定的无偏随机梯度的光滑非凸目标，我们建立的收敛率中主导随机项为𝒪(σ/√(NK))；当σ>0时，暂态时间对智能体数量、压缩水平和谱间隙Δ_λ的主导依赖关系为𝒪(N³/(δ⁴Δ_λ⁴))，与问题相关的固定因子被省略。在Polyak–Łojasiewicz条件下，CED-EF的主导随机项为𝓞̃(σ²/(NK))，暂态时间为𝓞̃(N/(δ²Δ_λ²))，这些依赖关系改善了现有结果的压缩和/或网络依赖特性。在最小二乘和逻辑回归问题上的数值实验表明了CED-EF的性能优势。
英文摘要
We study decentralized stochastic optimization over a network of $N$ agents under compressed communication. We propose CED-EF, an exact diffusion-based method with error feedback that directly accommodates biased $δ$-contractive compressors while communicating one compressed model-sized vector per node per iteration. For smooth nonconvex objectives with unbiased stochastic gradients whose variance is bounded by $σ^2$, where $σ\geq0$, we establish a convergence rate whose leading stochastic term is $\mathcal O(σ/\sqrt{NK})$. For $σ>0$, the dominant dependence of the corresponding transient time on the number of agents, compression level, and spectral gap $Δ_λ$ is $\mathcal O(N^3/(δ^4Δ_λ^4))$, with fixed problem-dependent factors suppressed. Under the Polyak--Łojasiewicz condition, CED-EF attains a leading stochastic term $\widetilde{\mathcal O}(σ^2/(NK))$ with transient time on the order of $\widetilde{\mathcal O}(N/(δ^2Δ_λ^2))$. These dependencies improve the compression and/or network dependence of existing results. Numerical experiments on least-squares and logistic-regression problems illustrate the performance advantages of CED-EF.
