Crowdsensing is a promising sensing paradigm for smart city applications (e.g., traffic and environment monitoring) with the prevalence of smart mobile devices and advanced network infrastructure. Meanwhile, as tasks are performed by individuals, privacy protection is one of the key issues in crowdsensing systems. Traditionally, to alleviate users' privacy concerns, noises are added to participants' sensitive data (e.g., participants' locations) through techniques such as differential privacy. However, this inevitably results in quality loss to the crowdsensing task. Recently, federated learning paradigm has been proposed, which aims to achieve privacy preservation in machine learning while ensuring that the learning quality suffers little or no loss. Inspired by the federated learning paradigm, this article studies how federated learning may benefit crowdsensing applications. In particular, we first propose a federated crowdsensing framework, which analyzes the privacy concerns of each crowdsensing stage (i.e., task creation, task assignment, task execution, and data aggregation) and discuss how federated learning techniques may take effect. Finally, we summarize key challenges and opportunities in federated crowdsensing.
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Mathematics and Computer Science, Fuzhou University（福州大学数学与计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Mobile crowdsensing (MCS) is an emerging sensing data collection pattern with scalability, low deployment cost, and distributed characteristics. Traditional MCS systems suffer from privacy concerns and fair reward distribution. Moreover, existing privacy-preserving MCS solutions usually focus on the privacy protection of data collection rather than that of data processing. To tackle faced problems of MCS, in this paper, we integrate federated learning (FL) into MCS and propose a privacy-preserving MCS system, called \textsc{CrowdFL}. Specifically, in order to protect privacy, participants loca…
Mobile Crowdsensing has become main stream paradigm for researchers to collect behavioral data from citizens in large scales. This valuable data can be leveraged to create centralized repositories that can be used to train advanced Artificial Intelligent (AI) models for various services that benefit society in all aspects. Although decades of research has explored the viability of Mobile Crowdsensing in terms of incentives and many attempts have been made to reduce the participation barriers, the overshadowing privacy concerns regarding sharing personal data still remain. Recently a new pathwa…
Xidian University（西安电子科技大学）；Fuzhou University（福州大学）；National Institute of Technology Kurukshetra（库鲁克谢特拉国家技术学院）
The application of federated extreme gradient boosting to mobile crowdsensing apps brings several benefits, in particular high performance on efficiency and classification. However, it also brings a new challenge for data and model privacy protection. Besides it being vulnerable to Generative Adversarial Network (GAN) based user data reconstruction attack, there is not the existing architecture that considers how to preserve model privacy. In this paper, we propose a secret sharing based federated learning architecture FedXGB to achieve the privacy-preserving extreme gradient boosting for mobi…
Federated learning is the process of developing machine learning models over datasets distributed across data centers such as hospitals, clinical research labs, and mobile devices while preventing data leakage. This survey examines previous research and studies on federated learning in the healthcare sector across a range of use cases and applications. Our survey shows what challenges, methods, and applications a practitioner should be aware of in the topic of federated learning. This paper aims to lay out existing research and list the possibilities of federated learning for healthcare indust…
Recently, Google and other 24 institutions proposed a series of open challenges towards federated learning (FL), which include application expansion and homomorphic encryption (HE). The former aims to expand the applicable machine learning models of FL. The latter focuses on who holds the secret key when applying HE to FL. For the naive HE scheme, the server is set to master the secret key. Such a setting causes a serious problem that if the server does not conduct aggregation before decryption, a chance is left for the server to access the user's update. Inspired by the two challenges, we pro…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
Federated Learning (FL) is a concept first introduced by Google in 2016, in which multiple devices collaboratively learn a machine learning model without sharing their private data under the supervision of a central server. This offers ample opportunities in critical domains such as healthcare, finance etc, where it is risky to share private user information to other organisations or devices. While FL appears to be a promising Machine Learning (ML) technique to keep the local data private, it is also vulnerable to attacks like other ML models. Given the growing interest in the FL domain, this …
Tianjin University（天津大学）；Hainan University（海南大学）；University of Macau（澳门大学）
Federated learning plays an important role in the process of smart cities. With the development of big data and artificial intelligence, there is a problem of data privacy protection in this process. Federated learning is capable of solving this problem. This paper starts with the current developments of federated learning and its applications in various fields. We conduct a comprehensive investigation. This paper summarize the latest research on the application of federated learning in various fields of smart cities. In-depth understanding of the current development of federated learning from…
Indiana University-Purdue University Indianapolis（印第安纳大学与普渡大学印第安纳波利斯联合分校）；Shandong University（山东大学）
Mobile crowdsensing (MCS) counting on the mobility of massive workers helps the requestor accomplish various sensing tasks with more flexibility and lower cost. However, for the conventional MCS, the large consumption of communication resources for raw data transmission and high requirements on data storage and computing capability hinder potential requestors with limited resources from using MCS. To facilitate the widespread application of MCS, we propose a novel MCS learning framework leveraging on blockchain technology and the new concept of edge intelligence based on federated learning (FL…
Federated learning methods typically learn a model by iteratively sampling updates from a population of clients. In this work, we explore how the number of clients sampled at each round (the cohort size) impacts the quality of the learned model and the training dynamics of federated learning algorithms. Our work poses three fundamental questions. First, what challenges arise when trying to scale federated learning to larger cohorts? Second, what parallels exist between cohort sizes in federated learning and batch sizes in centralized learning? Last, how can we design federated learning methods…
Carnegie Mellon University（卡内基梅隆大学）；Bosch Center for Artificial Intelligence（博世人工智能中心）；Determined AI
Federated learning involves training statistical models over remote devices or siloed data centers, such as mobile phones or hospitals, while keeping data localized. Training in heterogeneous and potentially massive networks introduces novel challenges that require a fundamental departure from standard approaches for large-scale machine learning, distributed optimization, and privacy-preserving data analysis. In this article, we discuss the unique characteristics and challenges of federated learning, provide a broad overview of current approaches, and outline several directions of future work …
Vertical Federated Learning: Concepts, Advances and Challenges
Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Webank（微众银行）；AsiaInfo Technologies（亚信科技）；Hong Kong University of Science and Technology（香港科技大学）
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Conventional machine learning techniques are conducted in a centralized manner. Recently, the massive volume of generated wireless data, the privacy concerns and the increasing computing capabilities of wireless end-devices have led to the emergence of a promising decentralized solution, termed as Wireless Federated Learning (WFL). In this first of the two parts paper, we present the application of WFL in the sixth generation of wireless networks (6G), which is envisioned to be an integrated communication and computing platform. After analyzing the key concepts of WFL, we discuss the core chal…
Federated learning (FL) has been developed as a promising framework to leverage the resources of edge devices, enhance customers' privacy, comply with regulations, and reduce development costs. Although many methods and applications have been developed for FL, several critical challenges for practical FL systems remain unaddressed. This paper provides an outlook on FL development, categorized into five emerging directions of FL, namely algorithm foundation, personalization, hardware and security constraints, lifelong learning, and nonstandard data. Our unique perspectives are backed by practic…
As the 5G communication networks are being widely deployed worldwide, both industry and academia have started to move beyond 5G and explore 6G communications. It is generally believed that 6G will be established on ubiquitous Artificial Intelligence (AI) to achieve data-driven Machine Learning (ML) solutions in heterogeneous and massive-scale networks. However, traditional ML techniques require centralized data collection and processing by a central server, which is becoming a bottleneck of large-scale implementation in daily life due to significantly increasing privacy concerns. Federated lea…
Federated learning is a distributed machine learning paradigm where multiple data owners (clients) collaboratively train one machine learning model while keeping data on their own devices. The heterogeneity of client datasets is one of the most important challenges of federated learning algorithms. Studies have found performance reduction with standard federated algorithms, such as FedAvg, on non-IID data. Many existing works on handling non-IID data adopt the same aggregation framework as FedAvg and focus on improving model updates either on the server side or on clients. In this work, we tac…
University of Vermont（佛蒙特大学）；MassMutual（万通互惠）；Raytheon（雷神公司）
Federated machine learning leverages edge computing to develop models from network user data, but privacy in federated learning remains a major challenge. Techniques using differential privacy have been proposed to address this, but bring their own challenges -- many require a trusted third party or else add too much noise to produce useful models. Recent advances in \emph{secure aggregation} using multiparty computation eliminate the need for a third party, but are computationally expensive especially at scale. We present a new federated learning protocol that leverages a novel differentially…
AIGC-assisted Federated Learning for Edge Intelligence: Architecture Design, Research Challenges and Future Directions
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Center for Intelligent Networking and Communications (CINC), University of Electronic Science and Technology of China（电子科技大学智能组网与通信中心）
Andalusian Research Institute in Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究院）；University of Granada（格拉纳达大学）
Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real…
Hong Kong University of Science and Technology（香港科技大学）；Webank（微众银行）；Beihang University（北京航空航天大学）
Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework…
Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, India（人工智能系，印度理工学院海得拉巴学院）；Department of Computing Technologies, Swinburne University of Technology, Hawthorn, australia（计算技术系，斯威本技术大学 Hawthorn 分校）；Department of Physics, Indian Institute of Technology Delhi, India（物理系，印度理工学院德里学院）；Department of Energy Science and Engineering, Indian Institute of Technology Delhi, India（能源科学与工程系，印度理工学院德里学院）；Department of Computer Science and Engineering, Indian Institute of Technology Hyderabad, India（计算机科学与工程系，印度理工学院海得拉巴学院）
Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（教育部新一代人工智能技术及其交叉应用重点实验室（东南大学））
Kennesaw State University（肯尼索州立大学）；Macquarie University（麦考瑞大学）；Brandon University（布兰登大学）；China Medical University（中国医药大学）
Federated Learning (FL) is known to perform Machine Learning tasks in a distributed manner. Over the years, this has become an emerging technology especially with various data protection and privacy policies being imposed FL allows performing machine learning tasks whilst adhering to these challenges. As with the emerging of any new technology, there are going to be challenges and benefits. A challenge that exists in FL is the communication costs, as FL takes place in a distributed environment where devices connected over the network have to constantly share their updates this can create a com…
University of Waterloo（滑铁卢大学）；University of Montreal（蒙特利尔大学）；Blue Lion Labs（蓝狮实验室）
Creating high-performance generalizable deep neural networks for phytoplankton monitoring requires utilizing large-scale data coming from diverse global water sources. A major challenge to training such networks lies in data privacy, where data collected at different facilities are often restricted from being transferred to a centralized location. A promising approach to overcome this challenge is federated learning, where training is done at site level on local data, and only the model parameters are exchanged over the network to generate a global model. In this study, we explore the feasibil…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Michigan State University（密歇根州立大学）
Federated learning (FL) emerges as a popular distributed learning schema that learns a model from a set of participating users without sharing raw data. One major challenge of FL comes with heterogeneous users, who may have distributionally different (or non-iid) data and varying computation resources. As federated users would use the model for prediction, they often demand the trained model to be robust against malicious attackers at test time. Whereas adversarial training (AT) provides a sound solution for centralized learning, extending its usage for federated users has imposed significant …
CSIRO（联邦科学与工业研究组织）；University of New South Wales（新南威尔士大学）
Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. A federated learning system can be viewed as a large-scale distributed system with different components and stakeholders as numerous client devices participate in federated learning. Designing a federated learning system requires software system design thinking apart from machine learning knowledge. Although much effort has been put into federated learning from the machine learning technique aspects, the software architecture design conc…
To address COVID-19 healthcare challenges, we need frequent sharing of health data, knowledge and resources at a global scale. However, in this digital age, data privacy is a big concern that requires the secure embedding of privacy assurance into the design of all technological solutions that use health data. In this paper, we introduce differential privacy by design (dPbD) framework and discuss its embedding into the federated machine learning system. To limit the scope of our paper, we focus on the problem scenario of COVID-19 imaging data privacy for disease diagnosis by computer vision an…
Technical University of Munich（慕尼黑工业大学）；University of Hamburg（汉堡大学）；Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Federated learning has faced performance and network communication challenges, especially in the environments where the data is not independent and identically distributed (IID) across the clients. To address the former challenge, we introduce the federated-centralized concordance property and show that the federated single-mini-batch training approach can achieve comparable performance as the corresponding centralized training in the Non-IID environments. To deal with the latter, we present the federated multi-mini-batch approach and illustrate that it can establish a trade-off between the pe…
From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcare
Imperial College London（伦敦帝国学院）；Massachusetts General Hospital（麻省总医院）；Zhejiang University（浙江大学）；Weill Cornell Medicine of Cornell University（康奈尔大学威尔医学院）；Royal Brompton Hospital（皇家布朗普顿医院）；King’s College London（伦敦国王学院）
Federated learning holds great promise in learning from fragmented sensitive data and has revolutionized how machine learning models are trained. This article provides a systematic overview and detailed taxonomy of federated learning. We investigate the existing security challenges in federated learning and provide a comprehensive overview of established defense techniques for data poisoning, inference attacks, and model poisoning attacks. The work also presents an overview of current training challenges for federated learning, focusing on handling non-i.i.d. data, high dimensionality issues, …
Vellore Institute of Technology（韦洛尔理工学院）；Charles Darwin University（查尔斯达尔文大学）；Stockholm University（斯德哥尔摩大学）；Chaitanya Bharathi Institute of Technology（柴塔尼亚巴拉蒂理工学院）
Internet of Underwater Things (IoUT) have gained rapid momentum over the past decade with applications spanning from environmental monitoring and exploration, defence applications, etc. The traditional IoUT systems use machine learning (ML) approaches which cater the needs of reliability, efficiency and timeliness. However, an extensive review of the various studies conducted highlight the significance of data privacy and security in IoUT frameworks as a predominant factor in achieving desired outcomes in mission critical applications. Federated learning (FL) is a secured, decentralized framew…
UC Santa Barbara（加州大学圣巴巴拉分校）；UT Austin（德克萨斯大学奥斯汀分校）；UPenn（宾夕法尼亚大学）；MIT（麻省理工学院）
Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-oriented challenges which include (i) communication bottleneck since a large number of devices upload their local updates to a parameter server, and (ii) scalability as the federated network consists of millions of devices. Due to these systems challenges as well as issues related to statistical heterogeneity of data and privacy concerns, designing a provably efficient federated learning method is of significant importance…
École Normale Supérieure（巴黎高等师范学院）；Technical University Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；Helmholtz Zentrum Muenchen（亥姆霍兹慕尼黑中心）
We propose a simple new aggregation strategy for federated learning that won the MICCAI Federated Tumor Segmentation Challenge 2021 (FETS), the first ever challenge on Federated Learning in the Machine Learning community. Our method addresses the problem of how to aggregate multiple models that were trained on different data sets. Conceptually, we propose a new way to choose the weights when averaging the different models, thereby extending the current state of the art (FedAvg). Empirical validation demonstrates that our approach reaches a notable improvement in segmentation performance compar…
Towards One-shot Federated Learning: Advances, Challenges, and Future Directions
Department of Electrical Engineering and Information Technology, University of Naples Federico II（那不勒斯费德里科二世大学电气工程与信息科技系）；Department of Mathematics and Applications "R. Caccioppoli", University of Naples Federico II（那不勒斯费德里科二世大学数学与应用系）；Department of Mathematics, Indian Institute of Technology Indore（印度理工学院Indore分校数学系）
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions.
Turku University of Applied Sciences（图尔库应用科学大学）；Riphah International University（里法哈国际大学）
In federated learning (FL), the global model at the server requires an efficient mechanism for weight aggregation and a systematic strategy for collaboration selection to manage and optimize communication payload. We introduce a practical and cost-efficient method for regularized weight aggregation and propose a laborsaving technique to select collaborators per round. We illustrate the performance of our method, regularized similarity weight aggregation (RegSimAgg), on the Federated Tumor Segmentation (FeTS) 2022 challenge's federated training (weight aggregation) problem. Our scalable approac…
There is a growing interest in the wireless communications community to complement the traditional model-based design approaches with data-driven machine learning (ML)-based solutions. While conventional ML approaches rely on the assumption of having the data and processing heads in a central entity, this is not always feasible in wireless communications applications because of the inaccessibility of private data and large communication overhead required to transmit raw data to central ML processors. As a result, decentralized ML approaches that keep the data where it is generated are much mor…
Billions of IoT devices will be deployed in the near future, taking advantage of faster Internet speed and the possibility of orders of magnitude more endpoints brought by 5G/6G. With the growth of IoT devices, vast quantities of data that may contain users' private information will be generated. The high communication and storage costs, mixed with privacy concerns, will increasingly challenge the traditional ecosystem of centralized over-the-cloud learning and processing for IoT platforms. Federated Learning (FL) has emerged as the most promising alternative approach to this problem. In FL, t…
Federated learning (FL) is an emerging technology that enables the training of machine learning models from multiple clients while keeping the data distributed and private. Based on the participating clients and the model training scale, federated learning can be classified into two types: cross-device FL where clients are typically mobile devices and the client number can reach up to a scale of millions; cross-silo FL where clients are organizations or companies and the client number is usually small (e.g., within a hundred). While existing studies mainly focus on cross-device FL, this paper …
Federated learning poses new statistical and systems challenges in training machine learning models over distributed networks of devices. In this work, we show that multi-task learning is naturally suited to handle the statistical challenges of this setting, and propose a novel systems-aware optimization method, MOCHA, that is robust to practical systems issues. Our method and theory for the first time consider issues of high communication cost, stragglers, and fault tolerance for distributed multi-task learning. The resulting method achieves significant speedups compared to alternatives in th…
