Decentralized learning (DL) enables collaborative learning without a server and without training data leaving the users' devices. However, the models shared in DL can still be used to infer training data. Conventional defenses such as differential privacy and secure aggregation fall short in effectively safeguarding user privacy in DL, either sacrificing model utility or efficiency. We introduce Shatter, a novel DL approach in which nodes create virtual nodes (VNs) to disseminate chunks of their full model on their behalf. This enhances privacy by (i) preventing attackers from collecting full models from other nodes, and (ii) hiding the identity of the original node that produced a given model chunk. We theoretically prove the convergence of Shatter and provide a formal analysis demonstrating how Shatter reduces the efficacy of attacks compared to when exchanging full models between nodes. We evaluate the convergence and attack resilience of Shatter with existing DL algorithms, with heterogeneous datasets, and against three standard privacy attacks. Our evaluation shows that Shatter not only renders these privacy attacks infeasible when each node operates 16 VNs but also exhibits a positive impact on model utility compared to standard DL. In summary, Shatter enhances the privacy of DL while maintaining the utility and efficiency of the model.
Decentralized Federated Learning: A Survey on Security and Privacy
University of Windsor(温莎大学);University of New Brunswick(新不伦瑞克大学);University of Western Ontario(西安大略大学);Hong Kong University of Science and Technology(香港科技大学);WeBank(微众银行)
Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization
CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Aalborg University(奥尔堡大学);University of Twente(特温特大学);Netherlands Defence Academy(荷兰国防学院);Delft University of Technology(代尔夫特理工大学)
D2M: A Decentralized, Privacy-Preserving, Incentive-Compatible Data Marketplace for Collaborative Learning
Department of Computer Science and Engineering, Indian Institute of Technology, Jodhpur(计算机科学与工程系,印度理工学院,乔浦尔);School of Mathematics and Computational Sciences, Eastern University Pennsylvania(数学与计算科学学院,东部大学宾夕法尼亚)
ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(计算能力网络与信息安全重点实验室,教育部,山东计算机科学中心,齐鲁大学(山东科学院));Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science(山东省工业网络与信息系统安全重点实验室,山东省计算机科学基础研究中心);School of Information and Software Engineering, University of Electronic Science and Technology of China(信息与软件工程学院,电子科技大学)
Towards Personalized Differentially Private Learning for Decentralized Local Graphs
School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学网络与交换技术国家重点实验室);School of Cyber Science and Technology, Shandong University(山东大学网络空间安全学院);School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络空间安全学院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)
Differentially Private Pre-Trained Model Fusion using Decentralized Federated Graph Matching
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Changsha Aier Eye Hospital(长沙爱尔眼科医院);Nanchang Aier Eye Hospital(南昌爱尔眼科医院)
Using Decentralized Aggregation for Federated Learning with Differential Privacy
College of Computing and Information Technology, Arab Academy for Science, Technology, and Maritime Transport (AASTMT)(阿拉伯科学、技术与海运学院(AASTMT)计算与信息技术学院);Universidade de Vigo(维戈大学)
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算中心(国家超级计算济南中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心)
Institute of Health Informatics, University College London(伦敦大学学院健康信息学研究所);Department of Psychiatry, University of Oxford(牛津大学精神病学系);Department of Diabetes & Endocrinology, University College London Hospitals(伦敦大学学院医院糖尿病与内分泌科);Department of Computer Science, University College London(伦敦大学学院计算机科学系)
Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning
Center for Data Science, Zhejiang University(浙江大学数据科学中心);College of Software, Zhejiang University(浙江大学软件学院);College of Computer Science, Zhejiang University(浙江大学计算机学院)
Secure Distributed/Federated Learning: Prediction-Privacy Trade-Off for Multi-Agent System
University of Southern California(南加州大学)
分散学习是一种有效的新兴范式,用于提高多个有界计算代理的计算能力。在大数据时代,在分布式和联邦学习(DL和FL)框架内执行推理,中央服务器需要处理大量数据,同时依赖各种代理执行多个分布式训练任务。考虑到分布式计算的拓扑结构,隐私已经成为一个首要的问题。此外,假设代理的信息处理能力有限,需要一个复杂的\textit{privacy-preserving decentralization}来确保有效的计算。为此,我们研究的\textit{privacy-aware server to multi-agent assignment}问题受到信息处理的限制与每个代理,同时保持隐私和确保学习信息的消息,通过分布式私人联邦学习(DPFL)的方法收到代理全球终端。为了找到一个分散的计划,两个代理系统,我们制定了一个优化问题,平衡隐私和准确性,考虑到与每个代理相关联的压缩约束的质量。我们提出了一个迭代收敛算法交替自洽方程。我们还对所提出的解决方案进行了数值评估,以显示隐私预测的权衡,并证明了新方法在确保DL和FL隐私方面的有效性。
School of Mathematics and Statistics, Beijing Jiaotong University(北京交通大学数学与统计学院);Department of Electrical and Electronic Engineering, Faculty of Engineering, Imperial College London(帝国理工学院工程学院电气与电子工程系)
On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Department of Electrical and Computer Engineering, University of California, San Diego(加利福尼亚大学圣地亚哥分校电气与计算机工程系)
SPDL: Blockchain-secured and Privacy-preserving Decentralized Learning
分散式学习涉及在远程移动设备、边缘服务器或云服务器上训练机器学习模型,同时保持数据本地化。尽管许多研究已经表明了保护隐私、提高训练性能或引入拜占庭弹性的可行性,但没有一个研究同时考虑了所有这些因素。因此,我们面临以下问题:\textit{how can we efficiently coordinate the decentralized learning process while simultaneously maintaining learning security and data privacy?}为了解决这个问题,在本文中,我们提出了SPDL,一个区块链安全和隐私保护的分散学习方案。SPDL将区块链、拜占庭容错(BFT)共识、BFT约束聚合规则(GAR)和差分隐私无缝集成到一个系统中,确保高效的机器学习,同时保持数据隐私、拜占庭容错、透明度和可追溯性。为了验证我们的计划,我们提供了严格的分析收敛性和遗憾的存在拜占庭节点。我们还建立了一个SPDL原型,并进行了广泛的实验,以证明SPDL是有效的,高效的强大的安全性和隐私保证。
Decentralized Federated Anomaly Detection in Smart Grids: A P2P Gossip Approach
Centre for Cyber Resilience and Trust (CREST), Deakin University(网络弹性与信任中心(CREST),迪肯大学);Department of Computer Science & IT, Latrobe University(计算机科学与信息技术系,拉筹伯大学)