Federated Learning is a framework that jointly trains a model \textit{with} complete knowledge on a remotely placed centralized server, but \textit{without} the requirement of accessing the data stored in distributed machines. Some work assumes that the data generated from edge devices are identically and independently sampled from a common population distribution. However, such ideal sampling may not be realistic in many contexts. Also, models based on intrinsic agency, such as active sampling schemes, may lead to highly biased sampling. So an imminent question is how robust Federated Learning is to biased sampling? In this work\footnote{\url{https://github.com/jiaqian/robustness_of_FL}}, we experimentally investigate two such scenarios. First, we study a centralized classifier aggregated from a collection of local classifiers trained with data having categorical heterogeneity. Second, we study a classifier aggregated from a collection of local classifiers trained by data through active sampling at the edge. We present evidence in both scenarios that Federated Learning is robust to data heterogeneity when local training iterations and communication frequency are appropriately chosen.
University of Nevada, Reno（内华达大学里诺分校）；IBM Research（IBM研究院）；University of Virginia（弗吉尼亚大学）
Data heterogeneity has been identified as one of the key features in federated learning but often overlooked in the lens of robustness to adversarial attacks. This paper focuses on characterizing and understanding its impact on backdooring attacks in federated learning through comprehensive experiments using synthetic and the LEAF benchmarks. The initial impression driven by our experimental results suggests that data heterogeneity is the dominant factor in the effectiveness of attacks and it may be a redemption for defending against backdooring as it makes the attack less efficient, more chal…
Google Research（谷歌研究院）；Carnegie Mellon University（卡内基梅隆大学）
Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metrics such as accuracy, fairness, and robustness in federated networks. However, despite a plethora of work in this area, it remains unclear: (1) which personalization techniques are most effective in various settings, and (2) how important personalization truly is for realistic federated applications. To better answer these questions, we propose Motley, a benchmark for personalized federated learning. Motley consists …
Hong Kong Baptist University（香港浸会大学）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Sydney（悉尼大学）；Tsinghua University（清华大学）
Privacy and security concerns in real-world applications have led to the development of adversarially robust federated models. However, the straightforward combination between adversarial training and federated learning in one framework can lead to the undesired robustness deterioration. We discover that the attribution behind this phenomenon is that the generated adversarial data could exacerbate the data heterogeneity among local clients, making the wrapped federated learning perform poorly. To deal with this problem, we propose a novel framework called Slack Federated Adversarial Training (…
Peking University（北京大学）；Central South University（中南大学）；University of Pennsylvania（宾夕法尼亚大学）；The University of Hong Kong（香港大学）；South China University of Technology（华南理工大学）；University of Maryland, Baltimore County (UMBC)（马里兰大学巴尔的摩县分校（UMBC））
Robust Federated Learning on Edge Devices with Domain Heterogeneity
Department of Computer Science and Engineering, Kyung Hee University, 171-04, Republic of Korea（韩国庆熙大学计算机科学与工程系）；Machine learning department, Mohamed Bin Zayed University of Artificial Intelligence, United Arab Emirates（阿联酋穆罕默德·本·扎耶德人工智能大学机器学习系）
Institute of Computing Technology, Chinese Academy of Sciences, China（中国科学院计算技术研究所）；Purple Mountain Laboratories, China（紫金山实验室）；University of Chinese Academy of Sciences, China（中国科学院大学）
Personalized Federated Dictionary Learning for Modeling Heterogeneity in Multi-site fMRI Data
Tulane Center for Biomedical Informatics and Genomics（Tulane生物医学信息学与基因组学中心）；School of Medicine, Tulane University（Tulane大学医学院）；State University of New York at Oswego（纽约州立大学奥尔巴尼分校）
An Architecture Built for Federated Learning: Addressing Data Heterogeneity through Adaptive Normalization-Free Feature Recalibration
Department of Computer Science, City St. George’s, University of London（伦敦大学城市圣乔治学院计算机科学系）；BioMedIA, Department of Computing, Imperial College London（伦敦帝国理工学院计算系BioMedIA）
On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）；La Trobe Business School, La Trobe University（拉筹伯大学拉筹伯商学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）；Hong Kong University of Science and Technology（香港科技大学）
FedXDS: Leveraging Model Attribution Methods to counteract Data Heterogeneity in Federated Learning
Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technical University of Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Yale University（耶鲁大学）
Federated learning (FL) algorithms usually sample a fraction of clients in each round (partial participation) when the number of participants is large and the server's communication bandwidth is limited. Recent works on the convergence analysis of FL have focused on unbiased client sampling, e.g., sampling uniformly at random, which suffers from slow wall-clock time for convergence due to high degrees of system heterogeneity and statistical heterogeneity. This paper aims to design an adaptive client sampling algorithm that tackles both system and statistical heterogeneity to minimize the wall-…
Privacy-Preserving Personalized Federated Learning for Distributed Photovoltaic Disaggregation under Statistical Heterogeneity
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Department of Data Science and AI, Faculty of IT and Monash Energy Institute, Monash University（墨尔本大学数据科学与人工智能系）；Shenzhen Institute for Advanced Study of UESTC（电子科技大学深圳研究院）
Stanford University（斯坦福大学）；The University of Hong Kong（香港大学）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
The collection and curation of large-scale medical datasets from multiple institutions is essential for training accurate deep learning models, but privacy concerns often hinder data sharing. Federated learning (FL) is a promising solution that enables privacy-preserving collaborative learning among different institutions, but it generally suffers from performance deterioration due to heterogeneous data distributions and a lack of quality labeled data. In this paper, we present a robust and label-efficient self-supervised FL framework for medical image analysis. Our method introduces a novel T…
Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity
School of Cyberspace Science and Technology, Beijing Institute of Technology（信息科学与技术学院，北京理工大学）；School of Information and Electronics, Beijing Institute of Technology（信息电子学院，北京理工大学）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）；Singapore University of Technology and Design（新加坡科技设计大学）；School of Cyber Science and Technology, Sun Yat-Sen University（网络科学与技术学院，中山大学）
Recently, local peer topology has been shown to influence the overall convergence of decentralized learning (DL) graphs in the presence of data heterogeneity. In this paper, we demonstrate the advantages of constructing a proxy-based locally heterogeneous DL topology to enhance convergence and maintain data privacy. In particular, we propose a novel peer clumping strategy to efficiently cluster peers before arranging them in a final training graph. By showing how locally heterogeneous graphs outperform locally homogeneous graphs of similar size and from the same global data distribution, we pr…
