Distributed learning, which does not require gathering training data in a central location, has become increasingly important in the big-data era. In particular, random-walk-based decentralized algorithms are flexible in that they do not need a central server trusted by all clients and do not require all clients to be active in all iterations. However, existing distributed learning algorithms assume that all learning clients share the same task. In this paper, we consider the more difficult meta-learning setting, in which different clients perform different (but related) tasks with limited training data. To reduce communication cost and allow better privacy protection, we propose LoDMeta (Local Decentralized Meta-learning) with the use of local auxiliary optimization parameters and random perturbations on the model parameter. Theoretical results are provided on both convergence and privacy analysis. Empirical results on a number of few-shot learning data sets demonstrate that LoDMeta has similar meta-learning accuracy as centralized meta-learning algorithms, but does not require gathering data from each client and is able to better protect data privacy for each client.
On the Tradeoff between Privacy Preservation and Byzantine-Robustness in Decentralized Learning
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Department of Electrical and Computer Engineering, University of California, San Diego(加利福尼亚大学圣地亚哥分校电气与计算机工程系)
Decentralized Matrix Factorization with Heterogeneous Differential Privacy
Research Institute for Interdisciplinary Sciences(交叉科学研究院);School of Information Management and Engineering(信息管理与工程学院);Shanghai University of Finance and Economics(上海财经大学)
ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(计算能力网络与信息安全重点实验室,教育部,山东计算机科学中心,齐鲁大学(山东科学院));Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science(山东省工业网络与信息系统安全重点实验室,山东省计算机科学基础研究中心);School of Information and Software Engineering, University of Electronic Science and Technology of China(信息与软件工程学院,电子科技大学)
Towards Personalized Differentially Private Learning for Decentralized Local Graphs
School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学网络与交换技术国家重点实验室);School of Cyber Science and Technology, Shandong University(山东大学网络空间安全学院);School of Cyberspace Security, Beijing University of Posts and Telecommunications(北京邮电大学网络空间安全学院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)
Decentralized Federated Learning: A Survey on Security and Privacy
University of Windsor(温莎大学);University of New Brunswick(新不伦瑞克大学);University of Western Ontario(西安大略大学);Hong Kong University of Science and Technology(香港科技大学);WeBank(微众银行)
Using Decentralized Aggregation for Federated Learning with Differential Privacy
College of Computing and Information Technology, Arab Academy for Science, Technology, and Maritime Transport (AASTMT)(阿拉伯科学、技术与海运学院(AASTMT)计算与信息技术学院);Universidade de Vigo(维戈大学)
Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization
CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Aalborg University(奥尔堡大学);University of Twente(特温特大学);Netherlands Defence Academy(荷兰国防学院);Delft University of Technology(代尔夫特理工大学)
Decentralized Federated Anomaly Detection in Smart Grids: A P2P Gossip Approach
Centre for Cyber Resilience and Trust (CREST), Deakin University(网络弹性与信任中心(CREST),迪肯大学);Department of Computer Science & IT, Latrobe University(计算机科学与信息技术系,拉筹伯大学)
PDSL: Privacy-Preserved Decentralized Stochastic Learning with Heterogeneous Data Distribution
School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东省计算中心(国家超级计算济南中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东省科学院));Shandong Fundamental Research Center for Computer Science(山东省计算机科学基础研究中心)
D2M: A Decentralized, Privacy-Preserving, Incentive-Compatible Data Marketplace for Collaborative Learning
Department of Computer Science and Engineering, Indian Institute of Technology, Jodhpur(计算机科学与工程系,印度理工学院,乔浦尔);School of Mathematics and Computational Sciences, Eastern University Pennsylvania(数学与计算科学学院,东部大学宾夕法尼亚)
Differentially Private Pre-Trained Model Fusion using Decentralized Federated Graph Matching
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Changsha Aier Eye Hospital(长沙爱尔眼科医院);Nanchang Aier Eye Hospital(南昌爱尔眼科医院)
School of Computer and Communication Sciences, EPFL(洛桑联邦理工学院计算机与通信科学学院);Université de Lille(里尔大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Centrale Lille(里尔中央理工学院);Inria, Univ Montpellier(法国国家信息与自动化研究所、蒙彼利埃大学)
Unified Privacy Guarantees for Decentralized Learning via Matrix Factorization
Inria, Université de Montpellier, INSERM Montpellier, France(法国蒙彼利埃大学、INSERM蒙彼利埃研究所、Inria);Institute of Science & Technology Austria (ISTA)(奥地利科学与技术研究所);University of Rennes, Inria, CNRS, IRISA(雷恩大学、Inria、CNRS、IRISA)
Lossless Privacy-Preserving Aggregation for Decentralized Federated Learning
Center for Data Science, Zhejiang University(浙江大学数据科学中心);College of Software, Zhejiang University(浙江大学软件学院);College of Computer Science, Zhejiang University(浙江大学计算机学院)