This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components, namely a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. Within each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, with the teacher and student having identical network structure. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that the proposed EFDLS achieves excellent performance on a set of selected UCR2018 datasets regarding top-1 accuracy.
Peak-Controlled Logits Poisoning Attack in Federated Distillation
School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机与信息技术学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems
Faculty of EEE, Phenikaa School of Engineering(电子工程系,Phenikaa工程学院);Phenikaa University(Phenikaa大学);VinUni-Illinois Smart Health Center(VinUniversity-伊利诺伊智能健康中心);VinUniversity(Vin大学);College of Engineering & Computer Science(工程与计算机科学学院)
FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification
Towards Personalized Federated Learning via Comprehensive Knowledge Distillation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
FedICT: Federated Multi-task Distillation for Multi-access Edge Computing
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhongguancun Laboratory(中关村实验室)
Jiangsu University(江苏大学);Southwest University(西南大学);The University of Sydney(悉尼大学);Monash University(莫纳什大学);Sichuan University(四川大学);Nanyang Technological University(南洋理工大学)
DeepFusion: Accelerating MoE Training via Federated Knowledge Distillation from Heterogeneous Edge Devices
School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦女王玛丽大学电子工程与计算机科学学院);Department of Computer Science, Faculty of Environment, Science and Economy, University of Exeter(埃克塞特大学环境、科学与经济学院计算机科学系);School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);Beijing Key Laboratory of Petroleum Data Mining, China University of Petroleum(北京石油数据挖掘重点实验室,中国石油大学)
Knowledge Distillation in Federated Edge Learning: A Survey
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Zhongguancun Laboratory(中关村实验室);Beijing Jiaotong University(北京交通大学)
Collaborative Innovation Center of Railway Traffic Safety, Beijing Jiaotong University(北京交通大学轨道交通运行安全协同创新中心);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
FedUKD: Federated UNet Model with Knowledge Distillation for Land Use Classification from Satellite and Street Views
Institute of High Performance Computing, A*STAR(新加坡科技研究局高性能计算研究所);Vellore Institute of Technology(韦洛尔理工学院);College of Engineering, Anna University(安娜大学工程学院);Measi Academy Of Architecture(米西建筑学院)
FedAL: Black-Box Federated Knowledge Distillation Enabled by Adversarial Learning
School of Information Engineering, Guangdong University of Technology(广东工业大学信息工程学院);School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院)
Adaptive Heterogeneous Compression for Resource-Efficient Federated Knowledge Distillation
School of Information Engineering, Guangdong University of Technology(广东工业大学信息工程学院);School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications(重庆邮电大学通信与信息工程学院)
Partial Knowledge Distillation for Alleviating the Inherent Inter-Class Discrepancy in Federated Learning
College of Informatics Huazhong Agricultural University(华中农业大学信息学院);Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology(香港科技大学电子与计算机工程系);NVIDIA Corporation(NVIDIA公司)
Federated Knowledge Distillation for Multi-Model Architectures Lithography Hotspot Detection
City University of New York(纽约城市大学);Sichuan Agricultural University(四川农业大学);The University of Sydney(悉尼大学);Shenzhen University of Advanced Technology(深圳先进技术大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of St Andrews(圣安德鲁大学);Southwest University(西南大学)