Decentralized learning (DL) is an emerging paradigm of collaborative machine learning that enables nodes in a network to train models collectively without sharing their raw data or relying on a central server. This paper introduces Zip-DL, a privacy-aware DL algorithm that leverages correlated noise to achieve robust privacy against local adversaries while ensuring efficient convergence at low communication costs. By progressively neutralizing the noise added during distributed averaging, Zip-DL combines strong privacy guarantees with high model accuracy. Its design requires only one communication round per gradient descent iteration, significantly reducing communication overhead compared to competitors. We establish theoretical bounds on both convergence speed and privacy guarantees. Moreover, extensive experiments demonstrating Zip-DL's practical applicability make it outperform state-of-the-art methods in the accuracy vs. vulnerability trade-off. Specifically, Zip-DL (i) reduces membership-inference attack success rates by up to 35% compared to baseline DL, (ii) decreases attack efficacy by up to 13% compared to competitors offering similar utility, and (iii) achieves up to 59% higher accuracy to completely nullify a basic attack scenario, compared to a state-of-the-art privacy-preserving approach under the same threat model. These results position Zip-DL as a practical and efficient solution for privacy-preserving decentralized learning in real-world applications.
On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Department of Electrical and Computer Engineering, University of California, San Diego(加利福尼亚大学圣地亚哥分校电气与计算机工程系)
School of Computer and Communication Sciences, EPFL(洛桑联邦理工学院计算机与通信科学学院);Université de Lille(里尔大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Centrale Lille(里尔中央理工学院);Inria, Univ Montpellier(法国国家信息与自动化研究所、蒙彼利埃大学)
ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(计算能力网络与信息安全重点实验室,教育部,山东计算机科学中心,齐鲁大学(山东科学院));Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science(山东省工业网络与信息系统安全重点实验室,山东省计算机科学基础研究中心);School of Information and Software Engineering, University of Electronic Science and Technology of China(信息与软件工程学院,电子科技大学)
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算中心(国家超级计算济南中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心)
Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization
CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Aalborg University(奥尔堡大学);University of Twente(特温特大学);Netherlands Defence Academy(荷兰国防学院);Delft University of Technology(代尔夫特理工大学)
Decentralized Matrix Factorization with Heterogeneous Differential Privacy
Research Institute for Interdisciplinary Sciences(交叉科学研究院);School of Information Management and Engineering(信息管理与工程学院);Shanghai University of Finance and Economics(上海财经大学)
Towards Personalized Differentially Private Learning for Decentralized Local Graphs
School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学网络与交换技术国家重点实验室);School of Cyber Science and Technology, Shandong University(山东大学网络空间安全学院);School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络空间安全学院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)
Decentralized Federated Learning: A Survey on Security and Privacy
University of Windsor(温莎大学);University of New Brunswick(新不伦瑞克大学);University of Western Ontario(西安大略大学);Hong Kong University of Science and Technology(香港科技大学);WeBank(微众银行)
Unified Privacy Guarantees for Decentralized Learning via Matrix Factorization
Inria, Université de Montpellier, INSERM Montpellier, France(法国蒙彼利埃大学、INSERM蒙彼利埃研究所、Inria);Institute of Science & Technology Austria (ISTA)(奥地利科学与技术研究所);University of Rennes, Inria, CNRS, IRISA(雷恩大学、Inria、CNRS、IRISA)
Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning
Center for Data Science, Zhejiang University(浙江大学数据科学中心);College of Software, Zhejiang University(浙江大学软件学院);College of Computer Science, Zhejiang University(浙江大学计算机学院)
Using Decentralized Aggregation for Federated Learning with Differential Privacy
College of Computing and Information Technology, Arab Academy for Science, Technology, and Maritime Transport (AASTMT)(阿拉伯科学、技术与海运学院(AASTMT)计算与信息技术学院);Universidade de Vigo(维戈大学)
Decentralized Federated Anomaly Detection in Smart Grids: A P2P Gossip Approach
Centre for Cyber Resilience and Trust (CREST), Deakin University(网络弹性与信任中心(CREST),迪肯大学);Department of Computer Science & IT, Latrobe University(计算机科学与信息技术系,拉筹伯大学)
D2M: A Decentralized, Privacy-Preserving, Incentive-Compatible Data Marketplace for Collaborative Learning
Department of Computer Science and Engineering, Indian Institute of Technology, Jodhpur(计算机科学与工程系,印度理工学院,乔浦尔);School of Mathematics and Computational Sciences, Eastern University Pennsylvania(数学与计算科学学院,东部大学宾夕法尼亚)
School of Mathematics and Statistics, Beijing Jiaotong University(北京交通大学数学与统计学院);Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London(帝国理工学院工程学院电气与电子工程系)