We study local stochastic gradient descent methods for solving federated optimization over a network of agents communicating indirectly through a centralized coordinator. We are interested in the Byzantine setting where there is a subset of $f$ malicious agents that could observe the entire network and send arbitrary values to the coordinator to disrupt the performance of other non-faulty agents. The objective of the non-faulty agents is to collaboratively compute the optimizer of their respective local functions under the presence of Byzantine agents. In this setting, prior works show that the local stochastic gradient descent method can only return an approximate of the desired solutions due to the impacts of Byzantine agents. Whether this method can find an exact solution remains an open question. In this paper, we will address this open question by proposing a new variant of the local stochastic gradient descent method. Under similar conditions that are considered in the existing works, we will show that the proposed method converges exactly to the desired solutions. We will provide theoretical results to characterize the convergence properties of our method, in particular, the proposed method convergences at an optimal rate $\mathcal{O}(1/k)$ in both strongly convex and non-convex settings, where $k$ is the number of iterations. Finally, we will present a number of simulations to illustrate our theoretical results.
Resilient Decentralized Preconditioned Stochastic Proximal Optimization with Gradient Sketching over Time-Varying Networks
Hubei Normal University（湖北师范大学）；City University of Hong Kong（香港城市大学）；University of New South Wales（新南威尔士大学）；Southwest University（西南大学）；Shenzhen University of Advanced Technology（深圳先进技术研究院）
Prox-DBRO-VR: A Unified Analysis on Byzantine-Resilient Decentralized Stochastic Composite Optimization with Variance Reduction and Non-Asymptotic Convergence Rates
Central South University（中南大学）；City University of Hong Kong（香港城市大学）；University of New South Wales（新南威尔士大学）；Southwest University（西南大学）
