Despite recent advancements in federated learning (FL), the integration of generative models into FL has been limited due to challenges such as high communication costs and unstable training in heterogeneous data environments. To address these issues, we propose PRISM, a FL framework tailored for generative models that ensures (i) stable performance in heterogeneous data distributions and (ii) resource efficiency in terms of communication cost and final model size. The key of our method is to search for an optimal stochastic binary mask for a random network rather than updating the model weights, identifying a sparse subnetwork with high generative performance; i.e., a ``strong lottery ticket''. By communicating binary masks in a stochastic manner, PRISM minimizes communication overhead. This approach, combined with the utilization of maximum mean discrepancy (MMD) loss and a mask-aware dynamic moving average aggregation method (MADA) on the server side, facilitates stable and strong generative capabilities by mitigating local divergence in FL scenarios. Moreover, thanks to its sparsifying characteristic, PRISM yields a lightweight model without extra pruning or quantization, making it ideal for environments such as edge devices. Experiments on MNIST, FMNIST, CelebA, and CIFAR10 demonstrate that PRISM outperforms existing methods, while maintaining privacy with minimal communication costs. PRISM is the first to successfully generate images under challenging non-IID and privacy-preserving FL environments on complex datasets, where previous methods have struggled.
Aalto University（阿尔托大学）；Columbia University（哥伦比亚大学）；Nokia Bell Labs（诺基亚贝尔实验室）；The University of Queensland（昆士兰大学）
Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks' rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communication network with a high load of transportation. This paper introduces two approaches for improving communication efficiency by dynamic sampling and top-$k$ selective masking. The former controls the f…
Federated Learning with Label-Masking Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, UCAS（中国科学院大学网络空间安全学院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）
University of British Columbia（不列颠哥伦比亚大学）；Purdue University（普渡大学）；Tianjin University（天津大学）
Federated learning (FL) is an effective technique to directly involve edge devices in machine learning training while preserving client privacy. However, the substantial communication overhead of FL makes training challenging when edge devices have limited network bandwidth. Existing work to optimize FL bandwidth overlooks downstream transmission and does not account for FL client sampling. In this paper we propose GlueFL, a framework that incorporates new client sampling and model compression algorithms to mitigate low download bandwidths of FL clients. GlueFL prioritizes recently used client…
SDBA: A Stealthy and Long-Lasting Durable Backdoor Attack in Federated Learning
Department of Information and Communication Engineering, Major in Information Security, Chosun University（信息与通信工程系，信息安全专业，全州大学）；AI Data Security Research Section, Electronics and Telecommunications Research Institute（人工智能数据安全研究部，电子电信研究院）；Department of Convergence Science, Kongju National University（融合科学系， Kongju国立大学）
Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs). Despite promising preliminary results, the trade-offs in the training of SNNs in a distributed scheme are not well understood. Here, we consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the clients without sharing any data. We investigate federated learning for training multiple SNNs at clients when two mechanisms reduce the uplink communication cost: i) random masking of the model updates sen…
Model compression is important in federated learning (FL) with large models to reduce communication cost. Prior works have been focusing on sparsification based compression that could desparately affect the global model accuracy. In this work, we propose a new scheme for upstream communication where instead of transmitting the model update, each client learns and transmits a light-weight synthetic dataset such that using it as the training data, the model performs similarly well on the real training data. The server will recover the local model update via the synthetic data and apply standard …
Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations
Data Science and Analytics Institute, University of Oklahoma（俄克拉荷马大学数据科学与分析学院）；School of Industrial and Systems Engineering, University of Oklahoma（俄克拉荷马大学工业与系统工程学院）；Isfahan University of Technology（伊斯法罕理工大学）
Masked Random Noise for Communication Efficient Federated Learning
Huazhong University of Science and Technology（华中科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；FiT, Tencent（腾讯FiT）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））
Perfectly-Private Analog Secure Aggregation in Federated Learning
Department of Electrical Engineering, Chalmers University of Technology, Sweden（瑞典查尔姆斯理工大学电子工程系）；Department of Mathematics and Systems Analysis, Aalto University, Finland（芬兰阿尔托大学数学与系统分析系）
The Capacity of Information-Theoretic Secure Aggregation in Federated Learning
Information Coding and Transmission (ICT) Key Laboratory of Sichuan Province, Southwest Jiaotong University（四川省信息编码与传输（ICT）重点实验室，西南交通大学）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
Federated and distributed learning applications for electronic health records and structured medical data: A scoping review
Duke-NUS Medical School（杜克-新加坡国立大学医学院）；National Dental Research Institute Singapore（新加坡国家牙科研究所）；National Dental Centre Singapore（新加坡国家牙科中心）；Oral Health Academic Clinical Programme, Duke-NUS Medical School（杜克-新加坡国立大学医学院口腔健康学术临床项目）；Programme in Health Services and Systems Research, Duke-NUS Medical School（杜克-新加坡国立大学医学院卫生服务与系统研究项目）；National University of Singapore（新加坡国立大学）；Duke University（杜克大学）；Singapore National Eye Centre（新加坡国家眼科中心）；Singapore Eye Research Institute（新加坡眼科研究所）；Imperial College London（伦敦帝国理工学院）；Singapore General Hospital（新加坡中央医院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究院）
PRISM: Distributed Inference for Foundation Models at Edge
Department of Electrical and Computer Engineering, Aarhus University（电子与计算机工程系，奥胡斯大学）；Faculty of Information Technology and Communication Sciences, Tampere University（信息科技与通信科学学院，塔尔库大学）
Communication-Efficient Byzantine-Robust Federated Conformal Prediction via Partial Sharing
Department of Electronic Systems, Norwegian University of Science and Technology（电子系统系，挪威科学技术大学）；Commonwealth Scientific and Industrial Research Organisation（澳大利亚联邦科学与工业研究组织）；Department of Information and Communications Engineering, Aalto University（信息与通信工程系，艾尔托大学）
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure. The situation gets even worse with distributed training on mobile devices (federated learning), which suffers from higher latency, lower throughput, and intermittent poor connections. In this paper, we find 99.9% of the gradient exchange in distributed SGD is redundant, and propose Deep Gradient Compression (DGC) to greatly reduce the communication bandwidth. To preserve accuracy dur…
Hong Kong University of Science and Technology（香港科技大学）；Nokia Bell Labs（诺基亚贝尔实验室）；University of Nevada, Reno（内华达大学里诺分校）
With the advancement of machine learning (ML) and its growing awareness, many organizations who own data but not ML expertise (data owner) would like to pool their data and collaborate with those who have expertise but need data from diverse sources to train truly generalizable models (model owner). In such collaborative ML, the data owner wants to protect the privacy of its training data, while the model owner desires the confidentiality of the model and the training method which may contain intellectual properties. However, existing private ML solutions, such as federated learning and split …
