With increasing concerns for data privacy and ownership, recent years have witnessed a paradigm shift in machine learning (ML). An emerging paradigm, federated learning (FL), has gained great attention and has become a novel design for machine learning implementations. FL enables the ML model training at data silos under the coordination of a central server, eliminating communication overhead and without sharing raw data. In this paper, we conduct a review of the FL paradigm and, in particular, compare the types, the network structures, and the global model aggregation methods. Then, we conducted a comprehensive review of FL applications in the energy domain (refer to the smart grid in this paper). We provide a thematic classification of FL to address a variety of energy-related problems, including demand response, identification, prediction, and federated optimizations. We describe the taxonomy in detail and conclude with a discussion of various aspects, including challenges, opportunities, and limitations in its energy informatics applications, such as energy system modeling and design, privacy, and evolution.
Differentially Private Federated Learning: A Systematic Review
Stevens Institute of Technology（史蒂文斯理工学院）；University of Connecticut（康涅狄格大学）；Tongji University（同济大学）；Northeastern University（东北大学）；The Hong Kong Polytechnic University（香港理工大学）；East China Normal University（华东师范大学）；Institute of Science Tokyo（东京科学研究所）
University of the Basque Country UPV/EHU（巴斯克大学（UPV/EHU））
Intrusion detection systems are evolving into intelligent systems that perform data analysis searching for anomalies in their environment. The development of deep learning technologies opened the door to build more complex and effective threat detection models. However, training those models may be computationally infeasible in most Internet of Things devices. Current approaches rely on powerful centralized servers that receive data from all their parties -- violating basic privacy constraints and substantially affecting response times and operational costs due to the huge communication overhe…
The Internet-of-Things (IoT) generates vast quantities of data, much of it attributable to individuals' activity and behaviour. Gathering personal data and performing machine learning tasks on this data in a central location presents a significant privacy risk to individuals as well as challenges with communicating this data to the cloud. However, analytics based on machine learning and in particular deep learning benefit greatly from large amounts of data to develop high-performance predictive models. This work reviews federated learning as an approach for performing machine learning on distr…
Beihang University（北京航空航天大学）；University of California, San Diego（加州大学圣地亚哥分校）
As an emerging technique, Federated Learning (FL) can jointly train a global model with the data remaining locally, which effectively solves the problem of data privacy protection through the encryption mechanism. The clients train their local model, and the server aggregates models until convergence. In this process, the server uses an incentive mechanism to encourage clients to contribute high-quality and large-volume data to improve the global model. Although some works have applied FL to the Internet of Things (IoT), medicine, manufacturing, etc., the application of FL is still in its infa…
Exploring Machine Learning Models for Federated Learning: A Review of Approaches, Performance, and Limitations
Data Science and Analytics Institute, University of Oklahoma（俄克拉荷马大学数据科学与分析学院）；School of Industrial and Systems Engineering, University of Oklahoma（俄克拉荷马大学工业与系统工程学院）；Isfahan University of Technology（伊斯法罕理工大学）
CSIRO（澳大利亚联邦科学与工业研究组织）；University of New South Wales（新南威尔士大学）
Federated learning is an emerging machine learning paradigm where clients train models locally and formulate a global model based on the local model updates. To identify the state-of-the-art in federated learning and explore how to develop federated learning systems, we perform a systematic literature review from a software engineering perspective, based on 231 primary studies. Our data synthesis covers the lifecycle of federated learning system development that includes background understanding, requirement analysis, architecture design, implementation, and evaluation. We highlight and summar…
Tianjin University（天津大学）；Hainan University（海南大学）；University of Macau（澳门大学）
Federated learning plays an important role in the process of smart cities. With the development of big data and artificial intelligence, there is a problem of data privacy protection in this process. Federated learning is capable of solving this problem. This paper starts with the current developments of federated learning and its applications in various fields. We conduct a comprehensive investigation. This paper summarize the latest research on the application of federated learning in various fields of smart cities. In-depth understanding of the current development of federated learning from…
Review of Mathematical Optimization in Federated Learning
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；National Engineering Laboratory for Big Data Analytics, Xi’an Jiaotong University（西安交通大学国家大数据分析工程实验室）
Tsinghua University（清华大学）；Fraunhofer Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹研究所）；RWTH Aachen University（亚琛工业大学）；A*STAR（新加坡科技研究局）；Keio University（庆应义塾大学）；Technical University of Berlin（柏林工业大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
The advent of Federated Learning (FL) has ignited a new paradigm for parallel and confidential decentralized Machine Learning (ML) with the potential of utilizing the computational power of a vast number of IoT, mobile and edge devices without data leaving the respective device, ensuring privacy by design. Yet, in order to scale this new paradigm beyond small groups of already entrusted entities towards mass adoption, the Federated Learning Framework (FLF) has to become (i) truly decentralized and (ii) participants have to be incentivized. This is the first systematic literature review analyzi…
Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, India（人工智能系，印度理工学院海得拉巴学院）；Department of Computing Technologies, Swinburne University of Technology, Hawthorn, australia（计算技术系，斯威本技术大学 Hawthorn 分校）；Department of Physics, Indian Institute of Technology Delhi, India（物理系，印度理工学院德里学院）；Department of Energy Science and Engineering, Indian Institute of Technology Delhi, India（能源科学与工程系，印度理工学院德里学院）；Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India（计算机科学与工程系，印度理工学院海得拉巴学院）
Multimodal Federated Learning in Healthcare: a Review
West Virginia University（西弗吉尼亚大学）；Tribhuvan University（特里布万大学）；University at Albany（奥尔巴尼大学）；University of Pittsburgh（匹兹堡大学）；University of Aberdeen（阿伯丁大学）
University of Murcia（穆尔西亚大学）；European Commission, Joint Research Centre（欧盟委员会联合研究中心）
The application of Machine Learning (ML) techniques to the well-known intrusion detection systems (IDS) is key to cope with increasingly sophisticated cybersecurity attacks through an effective and efficient detection process. In the context of the Internet of Things (IoT), most ML-enabled IDS approaches use centralized approaches where IoT devices share their data with data centers for further analysis. To mitigate privacy concerns associated with centralized approaches, in recent years the use of Federated Learning (FL) has attracted a significant interest in different sectors, including hea…
Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning
University of Helsinki（赫尔辛基大学）；University of Technology Sydney（悉尼科技大学）；Beihang University（北京航空航天大学）；Monash University（莫纳什大学）；Aalto University（阿尔托大学）；Griffith University（格里菲斯大学）；Amazon（亚马逊公司）；Columbia University（哥伦比亚大学）
University of Oxford（牛津大学）；Imperial College London（帝国理工学院）；McGill University（麦吉尔大学）
Federated learning (FL) is a promising way to allow multiple data owners (clients) to collaboratively train machine learning models without compromising data privacy. Yet, existing FL solutions usually rely on a centralized aggregator for model weight aggregation, while assuming clients are honest. Even if data privacy can still be preserved, the problem of single-point failure and data poisoning attack from malicious clients remains unresolved. To tackle this challenge, we propose to use distributed ledger technology (DLT) to achieve FLock, a secure and reliable decentralized Federated Learni…
Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；University of New South Wales（新南威尔士大学）
Federated learning is an emerging machine learning paradigm that enables multiple devices to train models locally and formulate a global model, without sharing the clients' local data. A federated learning system can be viewed as a large-scale distributed system, involving different components and stakeholders with diverse requirements and constraints. Hence, developing a federated learning system requires both software system design thinking and machine learning knowledge. Although much effort has been put into federated learning from the machine learning perspectives, our previous systematic…
From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcare
Imperial College London（伦敦帝国学院）；Massachusetts General Hospital（麻省总医院）；Zhejiang University（浙江大学）；Weill Cornell Medicine of Cornell University（康奈尔大学威尔医学院）；Royal Brompton Hospital（皇家布朗普顿医院）；King’s College London（伦敦国王学院）
Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review
Simula Metropolitan Center for Digital Engineering（西米拉数字工程中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Hospital Bonn（波恩大学医院）；Norwegian University of Science and Technology（挪威科技大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Birmingham City University（伯明翰城市大学）；ETH Zurich（苏黎世联邦理工学院）；Technical University of Munich（慕尼黑工业大学）
Recent Methodological Advances in Federated Learning for Healthcare
University of Cambridge（剑桥大学）；University of Bremen（不来梅大学）；University of Helsinki（赫尔辛基大学）；Addenbrooke’s Hospital, Cambridge University Hospitals NHS Trust（剑桥大学医院NHS信托基金会阿登布鲁克医院）
Northeastern University（东北大学）；The University of Melbourne（墨尔本大学）；RMIT University（皇家墨尔本理工大学）
Federated learning, which allows multiple client devices in a network to jointly train a machine learning model without direct exposure of clients' data, is an emerging distributed learning technique due to its nature of privacy preservation. However, it has been found that models trained with federated learning usually have worse performance than their counterparts trained in the standard centralized learning mode, especially when the training data is imbalanced. In the context of federated learning, data imbalance may occur either locally one one client device, or globally across many device…
Vertical Federated Learning: Concepts, Advances and Challenges
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Webank（微众银行）；AsiaInfo Technologies（亚信科技）；Hong Kong University of Science and Technology（香港科技大学）
Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we pro-vide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, cur-rent research work on handling chall…
Beijing University of Posts and Telecommunications（北京邮电大学）
Federated learning has emerged as an effective paradigm to achieve privacy-preserving collaborative learning among different parties. Compared to traditional centralized learning that requires collecting data from each party, in federated learning, only the locally trained models or computed gradients are exchanged, without exposing any data information. As a result, it is able to protect privacy to some extent. In recent years, federated learning has become more and more prevalent and there have been many surveys for summarizing related methods in this hot research topic. However, most of the…
A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendation
Australian Artificial Intelligence Institute, University of Technology Sydney（澳大利亚人工智能研究所，悉尼技术大学）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学数据科学与人工智能系）
Florida International University（佛罗里达国际大学）；Southern Illinois University（南伊利诺伊大学）
Federated Learning (FL) has gained widespread popularity in recent years due to the fast booming of advanced machine learning and artificial intelligence along with emerging security and privacy threats. FL enables efficient model generation from local data storage of the edge devices without revealing the sensitive data to any entities. While this paradigm partly mitigates the privacy issues of users' sensitive data, the performance of the FL process can be threatened and reached a bottleneck due to the growing cyber threats and privacy violation techniques. To expedite the proliferation of F…
Advancing oncology with federated learning: transcending boundaries in breast, lung, and prostate cancer. A systematic review
Seoul National University Bundang Hospital（首尔大学盆唐医院）；Maastricht University（马斯特里赫特大学）；Pfizer Pharma GmbH（辉瑞制药有限公司）；Queen Mary University of London（伦敦玛丽女王大学）；University of Oxford（牛津大学）；Imperial College London（帝国理工学院）；Flower Labs；Academy of Athens（雅典科学院）；Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿罗森多夫亥姆霍兹中心）；Pfizer Inc（辉瑞公司）
Specializing Directed Acyclic Graph Federated Learning(SDAGFL) is a new federated learning framework which updates model from the devices with similar data distribution through Directed Acyclic Graph Distributed Ledger Technology (DAG-DLT). SDAGFL has the advantage of personalization, resisting single point of failure and poisoning attack in fully decentralized federated learning. Because of these advantages, the SDAGFL is suitable for the federated learning in IoT scenario where the device is usually battery-powered. To promote the application of SDAGFL in IoT, we propose an energy optimized …
Federated learning (FL) is an emerging paradigm of collaborative machine learning that preserves user privacy while building powerful models. Nevertheless, due to the nature of open participation by self-interested entities, it needs to guard against potential misbehaviours by legitimate FL participants. FL verification techniques are promising solutions for this problem. They have been shown to effectively enhance the reliability of FL networks and help build trust among participants. Verifiable federated learning has become an emerging topic of research that has attracted significant interes…
FedReview: Review and Dispose Poisoned Updates without Validation Datasets or Historic Knowledge
Zhejiang University（浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究院）；ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC研究院）；University of Toronto（多伦多大学）
Macquarie University（麦考瑞大学）；University of New South Wales（新南威尔士大学）
Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy-preserving manner. The main challenges of FL is that end devices usually possess various computation and communication capabilities and their training data are not independent and identically distributed (non-IID). Due to limited communication bandwidth and unstable availability of such devices in a mobile network, only a fraction of end devices (also referred to as the participants or clients in a FL process) can be selected in each round. Hence, it is of paramount importan…
In this work, we demonstrate the viability of using federated learning to successfully predict energy consumption as well as solar production for all households within a certain network using low-power and low-space consuming embedded devices. We also demonstrate our prediction performance improving over time without the need for sharing private consumer energy data. We simulate a system with four nodes using data for one year to show this.
National University of Singapore（新加坡国立大学）；The University of Western Australia（西澳大利亚大学）
Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing systems and infrastructures to ease the development of various federated learning algorithms. Similar to deep learning systems such as PyTorch and TensorFlow that boost the development of deep learning, federated learning systems (FLSs) are equivalently important, and face challen…
Andalusian Research Institute in Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究院）；University of Granada（格拉纳达大学）
Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real…
Connected and Automated Vehicles (CAVs) are one of the emerging technologies in the automotive domain that has the potential to alleviate the issues of accidents, traffic congestion, and pollutant emissions, leading to a safe, efficient, and sustainable transportation system. Machine learning-based methods are widely used in CAVs for crucial tasks like perception, motion planning, and motion control, where machine learning models in CAVs are solely trained using the local vehicle data, and the performance is not certain when exposed to new environments or unseen conditions. Federated learning …
Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Software Engineering, Sun Yat-sen University（中山大学软件工程学院）；School of Systems Science and Engineering, Sun Yat-sen University（中山大学系统科学与工程学院）
Northeastern University（东北大学）；HKUST（香港科技大学）；Hong Kong Baptist University（香港浸会大学）
Federated learning utilizes various resources provided by participants to collaboratively train a global model, which potentially address the data privacy issue of machine learning. In such promising paradigm, the performance will be deteriorated without sufficient training data and other resources in the learning process. Thus, it is quite crucial to inspire more participants to contribute their valuable resources with some payments for federated learning. In this paper, we present a comprehensive survey of incentive schemes for federate learning. Specifically, we identify the incentive probl…
Quantum Federated Learning: A Comprehensive Survey
Department of Electrical and Computer Engineering, The University of Alabama in Huntsville, USA（电气与计算机工程系，美国阿拉巴马大学亨茨维尔分校）；Faculty of Engineering and Applied Science, Memorial University, Canada（工程与应用科学学院，加拿大纪念大学）；College of Computer Science and Engineering, Nanyang Technological University, Singapore（计算机科学与工程学院，新加坡南洋理工大学）
Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学）；Universidade da Coruña（拉科鲁尼亚大学）
Federated Learning is a novel framework that allows multiple devices or institutions to train a machine learning model collaboratively while preserving their data private. This decentralized approach is prone to suffer the consequences of data statistical heterogeneity, both across the different entities and over time, which may lead to a lack of convergence. To avoid such issues, different methods have been proposed in the past few years. However, data may be heterogeneous in lots of different ways, and current proposals do not always determine the kind of heterogeneity they are considering. …
Pennsylvania State University（宾夕法尼亚州立大学）；Fudan University（复旦大学）
Federated learning (FL) and split learning (SL) are two emerging collaborative learning methods that may greatly facilitate ubiquitous intelligence in Internet of Things (IoT). Federated learning enables machine learning (ML) models locally trained using private data to be aggregated into a global model. Split learning allows different portions of an ML model to be collaboratively trained on different workers in a learning framework. Federated learning and split learning, each has unique advantages and respective limitations, may complement each other toward ubiquitous intelligence in IoT. The…
