去中心化梯度下降：瓶颈机制与预算复杂度
Decentralized Gradient Descent: Bottleneck Regimes and Budget Complexity
浏览论文内容
中文总结 AI 辅助
研究去中心化梯度下降，开发以瓶颈为中心框架，引入DNR和GCR两个量，通过多阶段分析得出最优步长选择和预算复杂度界限，揭示各因素间权衡及预算分解情况。
中文摘要 AI 辅助
去中心化梯度下降（DGD）广泛用于解决代理网络上的分布式优化问题。虽然其收敛特性已为人熟知，但达到规定精度所需的通信和计算资源却了解较少。本文从资源感知角度研究DGD，刻画达到目标误差水平所需的通信 - 计算预算。我们开发了一个以瓶颈为中心的框架，其中不同因素在不同误差尺度下主导优化动态。具体确定了由初始化、目标异质性、网络连通性、梯度噪声和通信噪声控制的运行机制。引入两个基本量：梯度 - 多样性与网络连通性比率（DNR）和梯度与通信噪声比率（GCR）。表明这些量决定优化过程中遇到的瓶颈序列及相应的预算最优运行策略。通过多阶段分析，得出最优步长选择和明确的预算复杂度界限，揭示了整体预算如何分解为与连续瓶颈相关的贡献，并深入了解了目标异质性、网络连通性、梯度噪声和通信噪声之间的基本权衡。
英文摘要
Decentralized gradient descent (DGD) is widely used for solving distributed optimization problems over networks of agents. While its convergence properties are well understood, less is known about the communication and computation resources required to attain a prescribed accuracy. In this paper, we study DGD from a resource-aware perspective and characterize the communication-computation budget required to attain a target error level. We develop a bottleneck-centric framework in which different factors dominate the optimization dynamics at different error scales. Specifically, we identify operating regimes governed by initialization, objective heterogeneity and network connectivity, gradient noise, and communication noise. To capture these effects, we introduce two fundamental quantities: the gradient-Diversity-to-Network-connectivity Ratio (DNR) and the Gradient-to-Communication-noise Ratio (GCR). We show that these quantities determine the sequence of bottlenecks encountered during optimization and the corresponding budget-optimal operating strategy. Using a multi-stage analysis, we derive optimal stepsize selections and explicit budget-complexity bounds that quantify the budget resources required to attain a prescribed accuracy. The resulting expressions reveal how the overall budget decomposes into contributions associated with successive bottlenecks and provide insight into the fundamental tradeoffs among objective heterogeneity, network connectivity, gradient noise, and communication noise.
发表机构
- School of Electrical, Computer and Energy Engineering, Arizona State University（电气、计算机与能源工程学院，亚利桑那州立大学）
机构由 AI 辅助整理，请以论文原文为准。
