Federated learning (FL) enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other. However, it suffers the leakage of private information from uploading models. In addition, as the model size grows, the training latency increases due to limited transmission bandwidth and the model performance degrades while using differential privacy (DP) protection. In this paper, we propose a gradient sparsification empowered FL framework over wireless channels, in order to improve training efficiency without sacrificing convergence performance. Specifically, we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local training, thereby mitigating the performance degradation induced by DP and and reducing the number of transmission parameters over wireless channels. Then, we analyze the convergence bound of the proposed algorithm, by modeling a non-convex FL problem. Next, we formulate a time-sequential stochastic optimization problem for minimizing the developed convergence bound, under the constraints of transmit power, the average transmitting delay, as well as the client's DP requirement. Utilizing the Lyapunov drift-plus-penalty framework, we develop an analytical solution to the optimization problem. Extensive experiments have been implemented on three real life datasets to demonstrate the effectiveness of our proposed algorithm. We show that our proposed algorithms can fully exploit the interworking between communication and computation to outperform the baselines, i.e., random scheduling, round robin and delay-minimization algorithms.
Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
Shanghai Jiao Tong University(上海交通大学);Nanjing University of Science and Technology(南京理工大学);Data61, CSIRO(澳大利亚联邦科学与工业研究组织数据61研究所);Princeton University(普林斯顿大学)
Efficient Asynchronous Federated Learning with Sparsification and Quantization
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Hithink RoyalFlush Information Network Co., Ltd.(同花顺信息服务股份有限公司);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术及产业化协同创新中心);Department of Sciences and Informatics, Muroran Institute of Technology(室兰工业大学科学与信息学系);Boston Consulting Group(波士顿咨询公司)
Towards Communication-efficient Federated Learning via Sparse and Aligned Adaptive Optimization
Pillar of Information Systems Technology and Design(信息系统技术与设计学院);School of Information Science and Engineering(信息科学与工程学院);School of Cyber Science and Engineering(网络安全科学与工程学院);National Mobile Communications Research Laboratory(国家移动通信研究实验室);School of Electronic and Optical Engineering(电子与光学工程学院);School of Electronics, Electrical Engineering and Computer Science (EEECS)(电子、电气工程与计算机科学学院);Data61, CSIRO(Data61,CSIRO);Department of Electronics Engineering(电子工程系);Department of Electrical and Computer Engineering(电气与计算机工程系)
Dual-Segment Clustering Strategy for Hierarchical Federated Learning in Heterogeneous Wireless Environments
College of Electronics and Information Engineering, Tongji University(同济大学电子与信息工程学院);Tongji Hospital, Tongji University(同济大学附属同济医院);Data61, CSIRO(澳大利亚联邦科学与工业研究组织Data61研究院);Guangzhou Huatu Information Technology Co., Ltd.(广州华图信息技术有限公司);School of Electrical and Computer Engineering, The University of Sydney(悉尼大学电气与计算机工程学院)
Analysis and Optimization of Wireless Multimodal Federated Learning on Modal Heterogeneity
Shanghai Jiao Tong University(上海交通大学);Southeast University(东南大学);Data61, CSIRO(CSIRO数据61);Singapore University of Technology and Design(新加坡科技设计大学);Nanjing University of Science and Technology(南京理工大学);Jiangnan University(江南大学)
Joint Optimization for Federated Learning and Transmission over Unreliable Wireless Networks with Heterogeneous Data
Beijing University of Posts and Telecommunications(北京邮电大学);Jiaxing University(嘉兴学院);Northwestern Polytechnical University(西北工业大学);Southeast University(东南大学)
Lightweight Federated Learning over Wireless Edge Networks
Department of Electronic Engineering, Tsinghua University(电子工程系,清华大学);School of Cyber Science and Technology, Beihang University(信息科学与技术学院,北航);Tsinghua Space Center, Tsinghua University(清华大学空间中心);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
Adaptive Client Selection via Q-Learning-based Whittle Index in Wireless Federated Learning
Guangdong Prov./Zhuhai Key Lab of IRADS, Beijing Normal-Hong Kong Baptist University, Guangdong, P. R. China(广东/珠海IRADS实验室,北京师范大学-香港 Baptist大学,广东,中华人民共和国);College of Engineering, Carnegie Mellon University, Pittsburgh, PA, USA(工程学院,卡内基梅隆大学,匹兹堡,PA,美国);Department of Statistics, University of Oxford, 24-29 St Giles', Oxford OX1 3LB, U.K.(统计学系,牛津大学,牛津OX1 3LB,英国)
Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation
The University of Sydney(悉尼大学);EPFL(洛桑联邦理工学院);Kyung Hee University(庆熙大学);Vietnam - Korea University of Information and Communication Technology, The University of Danang(岘港大学越韩信息与通信技术大学)