Despite achieving remarkable performance, Federated Learning (FL) suffers from two critical challenges, i.e., limited computational resources and low training efficiency. In this paper, we propose a novel FL framework, i.e., FedDUAP, with two original contributions, to exploit the insensitive data on the server and the decentralized data in edge devices to further improve the training efficiency. First, a dynamic server update algorithm is designed to exploit the insensitive data on the server, in order to dynamically determine the optimal steps of the server update for improving the convergence and accuracy of the global model. Second, a layer-adaptive model pruning method is developed to perform unique pruning operations adapted to the different dimensions and importance of multiple layers, to achieve a good balance between efficiency and effectiveness. By integrating the two original techniques together, our proposed FL model, FedDUAP, significantly outperforms baseline approaches in terms of accuracy (up to 4.8% higher), efficiency (up to 2.8 times faster), and computational cost (up to 61.9% smaller).
Efficient Federated Learning Using Dynamic Update and Adaptive Pruning with Momentum on Shared Server Data
Hithink RoyalFlush Information Network Co., Ltd.(同花顺网络信息股份有限公司);School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Collaborative Innovation Center of Novel Software Technology and Industrialization(新型软件技术与产业化协同创新中心);Baidu Research(百度研究院);Software Institute, Peking University(北京大学软件学院);Auburn University(奥本大学);North Carolina State University(北卡罗来纳州立大学);School of Computer Science, Fudan University(复旦大学计算机科学学院)
Pruning-Aware Multi-Cluster Co-Inference for Large AI Models in AI-RANs
Shenzhen University(深圳大学);Guangdong Provincial Key Laboratory of Future Networks of Intelligence(广东省未来网络智能重点实验室);The Hang Seng University of Hong Kong(香港恒生大学);Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)(深圳未来网络智能研究院);ShanghaiTech University(上海科技大学);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shenzhen Research Institute of Big Data(深圳市大数据研究院);The University of Hong Kong(香港大学)
Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computing
Tsinghua University(清华大学);JD Explore Academy(京东探索研究院);Hong Kong Polytechnic University(香港理工大学);Nanjing University of Science and Technology(南京理工大学);Shenzhen Wisdom and Strategy Technology Co., Ltd.(深圳智略科技有限公司)
Adaptive Graph Pruning with Sudden-Events Evaluation for Traffic Prediction using Online Semi-Decentralized ST-GNNs
University of Zagreb, Faculty of Electrical Engineering and Computing(Zagreb大学电子工程与计算学院);RISE Research Institutes of Sweden(瑞典RISE研究机构);KTH Royal Institute of Technology(皇家理工学院)
Model Pruning Enables Efficient Federated Learning on Edge Devices
Yale University(耶鲁大学);IBM T. J. Watson Research Center(IBM托马斯·J·沃森研究中心);Stanford Institute for Human-Centered Artificial Intelligence(斯坦福人类中心人工智能研究所);Imperial College London(帝国理工学院)
Knowledge-Enhanced Semi-Supervised Federated Learning for Aggregating Heterogeneous Lightweight Clients in IoT
Pennsylvania State University(宾夕法尼亚州立大学);University of Electronic Science and Technology of China(电子科技大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Purdue University(普渡大学);Georgia State University(佐治亚州立大学)
Joint Optimization for Federated Learning and Transmission over Unreliable Wireless Networks with Heterogeneous Data
Beijing University of Posts and Telecommunications(北京邮电大学);Jiaxing University(嘉兴学院);Northwestern Polytechnical University(西北工业大学);Southeast University(东南大学)