Byzantine-Robust and Communication-Efficient Distributed Learning via Compressed Momentum Filtering
Digital Futures（数字未来机构）；College of Computer Science, Sichuan University（四川大学计算机学院）；Institute of Clinical Pathology, West China Hospital, Sichuan University（四川大学华西医院临床病理研究所）
Changxin Liu, Yanghao Li, Yuhao Yi, Karl H. Johansson
Distributed learning has become the standard approach for training large-scale machine learning models across private data silos. While distributed learning enhances privacy preservation and training efficiency, it faces critical challenges related to Byzantine robustness and communication reduction. Existing Byzantine-robust and communication-efficient methods rely on full gradient information either at every iteration or at certain iterations with a probability, and they only converge to an unnecessarily large neighborhood around the solution. Motivated by these issues, we propose a novel Byzantine-robust and communication-efficient stochastic distributed learning method that imposes no requirements on batch size and converges to a smaller neighborhood around the optimal solution than all existing methods, aligning with the theoretical lower bound. Our key innovation is leveraging Polyak Momentum to mitigate the noise caused by both biased compressors and stochastic gradients, thus defending against Byzantine workers under information compression. We provide proof of tight complexity bounds for our algorithm in the context of non-convex smooth loss functions, demonstrating that these bounds match the lower bounds in Byzantine-free scenarios. Finally, we validate the practical significance of our algorithm through an extensive series of experiments, benchmarking its performance on both binary classification and image classification tasks.
Screen Before You Fetch: Compressed Byzantine Screening for Decentralized Learning on the Edge-Cloud Continuum
School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Department of Information and Computer Science, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学信息与计算机科学系）
Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedback
Halicioglu Data Science Institute, UC San Diego（哈利乔格鲁数据科学研究所，加州大学圣迭戈分校）；UC Berkeley（加州大学伯克利分校）；UMass Amherst（马萨诸塞大学阿默斯特分校）；EECS department, UC Berkeley（加州大学伯克利分校电子工程与计算机科学系）
Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning
College of Computer Science, Nankai University（南开大学计算机科学学院）；School of Mathematical Sciences, Nankai University（南开大学数学科学学院）；College of Cyber Science, Nankai University（南开大学网络安全学院）；Department of Computer Science and Engineering, University of Louisville（路易斯维尔大学计算机科学与工程系）；Department of Computer Science and Engineering, University of North Texas（北德克萨斯大学计算机科学与工程系）
Byzantine-tolerant distributed learning of finite mixture models
Institute of Statistics and Big Data, Renmin University of China, Beijing, China（统计与大数据研究院，中国人民大学，北京，中国）；Department of Statistics and Data Science, National University of Singapore, Singapore（统计与数据科学系，新加坡国立大学，新加坡）；Department of Statistics, The University of British Columbia, Vancouver, Canada（统计系，不列颠哥伦比亚大学，温哥华，加拿大）
H+: An Efficient Similarity-Aware Aggregation for Byzantine Resilient Federated Learning
Beijing Institute of Technology（北京理工大学）；Southwest University（西南大学）；Sun Yat-Sen University（中山大学）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
Communication-Efficient Byzantine-Robust Federated Conformal Prediction via Partial Sharing
Department of Electronic Systems, Norwegian University of Science and Technology（电子系统系，挪威科学技术大学）；Commonwealth Scientific and Industrial Research Organisation（澳大利亚联邦科学与工业研究组织）；Department of Information and Communications Engineering, Aalto University（信息与通信工程系，艾尔托大学）
Simeon -- Secure Federated Machine Learning Through Iterative Filtering
School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；Department of Computer Science, Purdue University（普渡大学计算机科学系）
