Prox-DBRO-VR：基于容错的去中心化随机复合优化统一分析
Prox-DBRO-VR: A Unified Analysis on Byzantine-Resilient Decentralized Stochastic Composite Optimization with Variance Reduction and Non-Asymptotic Convergence Rates
AI总结：
本文提出Prox-DBRO-VR算法，通过结合鲁棒聚合机制和近端梯度映射方法，实现拜占庭鲁棒的去中心化随机复合优化，在标准假设下保证线性或亚线性收敛，并通过实验验证其有效性。
摘要：
去中心化随机梯度算法在所有网络节点可靠时能够高效解决大规模有限和优化问题。然而，大多数这些算法在面对故障节点、软件错误和网络攻击等不利条件时缺乏鲁棒性。本文旨在处理多智能体系统（MASs）中的一类通用复合优化问题，在存在未知数量的拜占庭节点的情况下。基于鲁棒聚合机制和近端梯度映射方法，提出了一种拜占庭鲁棒的去中心化随机近端梯度算法框架，称为Prox-DBRO-VR，仅利用本地计算和通信实现优化和控制目标。为了渐近减少本地梯度估计引起的噪声方差并加速收敛，我们将在Prox-DBRO-VR中加入两种局部方差减少（VR）技术（SAGA和LSVRG）来设计Prox-DBRO-SAGA和Prox-DBRO-LSVRG。通过在统一的理论框架中分析梯度学习误差、鲁棒一致性条件和收敛误差之间的收缩关系，证明在标准假设下，Prox-DBRO-SAGA和Prox-DBRO-LSVRG在最优解周围的一个误差球内，通过精心设计的常数（或衰减）步长，能够线性（或亚线性）收敛。同时，还 characterized 在线性和亚线性情况下的收敛精度与拜占庭鲁棒性之间的权衡。在数值实验中，通过在各种拜占庭攻击下解决去中心化稀疏机器学习问题，验证了所提算法的有效性和实用性。
英文摘要：
Decentralized stochastic gradient algorithms efficiently solve large-scale finite-sum optimization problems when all agents in the network are reliable. However, most of these algorithms are not resilient to adverse conditions, such as malfunctioning agents, software bugs, and cyber attacks. This paper aims to handle a class of general composite optimization problems over multi-agent systems (MASs) in the presence of an unknown number of Byzantine agents. Building on a resilient aggregation mechanism and the proximal-gradient mapping method, a Byzantine-resilient decentralized stochastic proximal-gradient algorithmic framework is proposed, dubbed Prox-DBRO-VR, which achieves an optimization and control goal using only local computations and communications. To asymptotically reduce the noise variance arising from local gradient estimation and accelerate the convergence, we incorporate two localized variance-reduced (VR) techniques (SAGA and LSVRG) into Prox-DBRO-VR to design Prox-DBRO-SAGA and Prox-DBRO-LSVRG. By analyzing the contraction relationships among the gradient-learning error, resilient consensus condition, and convergence error in a unified theoretical framework, it is proved that both Prox-DBRO-SAGA and Prox-DBRO-LSVRG, with a well-designed constant (resp., decaying) step-size, converge linearly (resp., sub-linearly) inside an error ball around the optimal solution to the original problem under standard assumptions. A trade-off between convergence accuracy and Byzantine resilience in both linear and sub-linear cases is also characterized. In numerical experiments, the effectiveness and practicability of the proposed algorithms are manifested via resolving a decentralized sparse machine-learning problem under various Byzantine attacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
