In Machine Learning scenarios, privacy is a crucial concern when models have to be trained with private data coming from users of a service, such as a recommender system, a location-based mobile service, a mobile phone text messaging service providing next word prediction, or a face image classification system. The main issue is that, often, data are collected, transferred, and processed by third parties. These transactions violate new regulations, such as GDPR. Furthermore, users usually are not willing to share private data such as their visited locations, the text messages they wrote, or the photo they took with a third party. On the other hand, users appreciate services that work based on their behaviors and preferences. In order to address these issues, Federated Learning (FL) has been recently proposed as a means to build ML models based on private datasets distributed over a large number of clients, while preventing data leakage. A federation of users is asked to train a same global model on their private data, while a central coordinating server receives locally computed updates by clients and aggregate them to obtain a better global model, without the need to use clients' actual data. In this work, we extend the FL approach by pushing forward the state-of-the-art approaches in the aggregation step of FL, which we deem crucial for building a high-quality global model. Specifically, we propose an approach that takes into account a suite of client-specific criteria that constitute the basis for assigning a score to each client based on a priority of criteria defined by the service provider. Extensive experiments on two publicly available datasets indicate the merits of the proposed approach compared to standard FL baseline.
With the wealth of information produced by social networks, smartphones, medical or financial applications, speculations have been raised about the sensitivity of such data in terms of users' personal privacy and data security. To address the above issues, Federated Learning (FL) has been recently proposed as a means to leave data and computational resources distributed over a large number of nodes (clients) where a central coordinating server aggregates only locally computed updates without knowing the original data. In this work, we extend the FL framework by pushing forward the state the ar…
Lebanese American University（黎巴嫩美国大学）；Concordia University（康考迪亚大学）；New York University（纽约大学）；Zayed University（扎耶德大学）；Khalifa University（哈利法大学）；Mohammad Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
In this paper, we increase the availability and integration of devices in the learning process to enhance the convergence of federated learning (FL) models. To address the issue of having all the data in one location, federated learning, which maintains the ability to learn over decentralized data sets, combines privacy and technology. Until the model converges, the server combines the updated weights obtained from each dataset over a number of rounds. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existin…
University College London（伦敦大学学院）；Duke University（杜克大学）；Apple Inc.（苹果公司）
Federated learning is an increasingly popular paradigm that enables a large number of entities to collaboratively learn better models. In this work, we study minimax group fairness in federated learning scenarios where different participating entities may only have access to a subset of the population groups during the training phase. We formally analyze how our proposed group fairness objective differs from existing federated learning fairness criteria that impose similar performance across participants instead of demographic groups. We provide an optimization algorithm -- FedMinMax -- for so…
FedCau: A Proactive Stop Policy for Communication and Computation Efficient Federated Learning
School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology（瑞典皇家理工学院电气与计算机科学学院）；Ericsson（爱立信公司）；Uppsala University（乌普萨拉大学）
University College London（伦敦大学学院）；Duke University（杜克大学）
Federated learning is an increasingly popular paradigm that enables a large number of entities to collaboratively learn better models. In this work, we study minmax group fairness in paradigms where different participating entities may only have access to a subset of the population groups during the training phase. We formally analyze how this fairness objective differs from existing federated learning fairness criteria that impose similar performance across participants instead of demographic groups. We provide an optimization algorithm -- FedMinMax -- for solving the proposed problem that pr…
Distributed optimization: designed for federated learning
School of Management（管理学院）；Jinan University（暨南大学）；Guangdong International Cooperation Base of Science and Technology for GBA Smart Logistics（粤港澳大湾区智能物流科技国际合作基地）；School of Intelligent Systems Science and Engineering（智能系统科学与工程学院）；Institute of Physical Internet（物理互联网研究院）；School of Mechanical and Electrical Engineering（机械与电子工程学院）；Jiangxi University of Science and Technology（江西理工大学）；Department of Industrial and Systems Engineering（工业与系统工程系）；The Hong Kong Polytechnic University（香港理工大学）
Federated Learning (FL) is an emerging direction in distributed machine learning (ML) that enables in-situ model training and testing on edge data. Despite having the same end goals as traditional ML, FL executions differ significantly in scale, spanning thousands to millions of participating devices. As a result, data characteristics and device capabilities vary widely across clients. Yet, existing efforts randomly select FL participants, which leads to poor model and system efficiency. In this paper, we propose Oort to improve the performance of federated training and testing with guided par…
Xidian University（西安电子科技大学）；JD Tech（京东科技）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；JD Finance America Corporation（京东金融美国公司）；University of Pittsburgh（匹兹堡大学）
Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. A complete list of metrics to evaluate VFL algorithms should include model applicability, privacy security, communication cost, and computation efficiency, where privacy security is especially important to VFL. However, to the best of our knowledge, there does not exist a VFL algorithm satisfying all these criteria very well. To address this challenging problem, in this paper, we reveal that zeroth-order optimization (ZOO) is a desir…
Prioritizing Gradient Sign Over Modulus: An Importance-Aware Framework for Wireless Federated Learning
National Mobile Communications Research Laboratory (NCRL)（国家移动通信研究中心（NCRL））；Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）；Zhejiang Lab（浙江实验室）；College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki（希腊亚里士多德大学电子与计算机工程系）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Recent Methodological Advances in Federated Learning for Healthcare
University of Cambridge（剑桥大学）；University of Bremen（不来梅大学）；University of Helsinki（赫尔辛基大学）；Addenbrooke’s Hospital, Cambridge University Hospitals NHS Trust（剑桥大学医院NHS信托基金会阿登布鲁克医院）
Australian Artificial Intelligence Institute, University of Technology Sydney（悉尼科技大学澳大利亚人工智能研究所）；University of Washington, Seattle（西雅图华盛顿大学）；University of Maryland, College Park（马里兰大学帕克分校）；Hong Kong Baptist University（香港浸会大学）
Federated learning (FL) aims at training a global model on the server side while the training data are collected and located at the local devices. Hence, the labels in practice are usually annotated by clients of varying expertise or criteria and thus contain different amounts of noises. Local training on noisy labels can easily result in overfitting to noisy labels, which is devastating to the global model through aggregation. Although recent robust FL methods take malicious clients into account, they have not addressed local noisy labels on each device and the impact to the global model. In …
Huazhong University of Science and Technology（华中科技大学）；Hubei University（湖北大学）；Simon Fraser University（西蒙弗雷泽大学）
Federated learning (FL) has recently emerged as a promising distributed machine learning (ML) paradigm. Practical needs of the "right to be forgotten" and countering data poisoning attacks call for efficient techniques that can remove, or unlearn, specific training data from the trained FL model. Existing unlearning techniques in the context of ML, however, are no longer in effect for FL, mainly due to the inherent distinction in the way how FL and ML learn from data. Therefore, how to enable efficient data removal from FL models remains largely under-explored. In this paper, we take the first…
Utah State University（犹他州立大学）；University of Wisconsin-Whitewater（威斯康星大学怀特沃特分校）
A new machine learning (ML) technique termed as federated learning (FL) aims to preserve data at the edge devices and to only exchange ML model parameters in the learning process. FL not only reduces the communication needs but also helps to protect the local privacy. Although FL has these advantages, it can still experience large communication latency when there are massive edge devices connected to the central parameter server (PS) and/or millions of model parameters involved in the learning process. Over-the-air computation (AirComp) is capable of computing while transmitting data by allowi…
National Institute of Posts and Telecommunications（国家邮电学院）
Federated Machine Learning (Fed ML) is a new distributed machine learning technique applied to collaboratively train a global model using clients local data without transmitting it. Nodes only send parameter updates (e.g., weight updates in the case of neural networks), which are fused together by the server to build the global model. By not divulging node data, Fed ML guarantees its confidentiality, a crucial aspect of network security, which enables it to be used in the context of data-sensitive Internet of Things (IoT) and mobile applications, such as smart Geo-location and the smart grid. …
Institute of Statistics and Big Data（统计与大数据研究院）；Renmin University of China（中国人民大学）；MetaAI（元宇宙人工智能实验室）；The University of British Columbia（不列颠哥伦比亚大学）；RIKEN（理化学研究所）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Purdue University（普渡大学）；Arizona State University（亚利桑那州立大学）；NC State University（北卡罗来纳州立大学）
Federated learning has generated significant interest, with nearly all works focused on a "star" topology where nodes/devices are each connected to a central server. We migrate away from this architecture and extend it through the network dimension to the case where there are multiple layers of nodes between the end devices and the server. Specifically, we develop multi-stage hybrid federated learning (MH-FL), a hybrid of intra- and inter-layer model learning that considers the network as a multi-layer cluster-based structure. MH-FL considers the topology structures among the nodes in the clus…
Interplay between Federated Learning and Explainable Artificial Intelligence: a Scoping Review
Simula Metropolitan Center for Digital Engineering（西米拉数字工程中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Hospital Bonn（波恩大学医院）；Norwegian University of Science and Technology（挪威科技大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Birmingham City University（伯明翰城市大学）；ETH Zurich（苏黎世联邦理工学院）；Technical University of Munich（慕尼黑工业大学）
Hong Kong Polytechnic University（香港理工大学）；Dalian University of Technology（大连理工大学）
We explore the problem of selectively forgetting categories from trained CNN classification models in the federated learning (FL). Given that the data used for training cannot be accessed globally in FL, our insights probe deep into the internal influence of each channel. Through the visualization of feature maps activated by different channels, we observe that different channels have a varying contribution to different categories in image classification. Inspired by this, we propose a method for scrubbing the model clean of information about particular categories. The method does not require …
A Comprehensive Data-centric Overview of Federated Graph Learning
Beijing Institute of Technology（北京理工大学）；Sun Yat-sen University（中山大学）；Peking University（北京大学）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）
FedFog: Network-Aware Optimization of Federated Learning over Wireless Fog-Cloud Systems
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg（卢森堡大学安全、可靠性与信任跨学科中心（SnT））；School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast（贝尔法斯特女王大学电子、电气工程与计算机科学学院）
History-Aware and Dynamic Client Contribution in Federated Learning
Max Planck Institute for Software Systems, Germany（马克斯·普朗克软件系统研究所，德国）；Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, France（里尔大学，法国）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore（高性能计算研究所（IHPC），新加坡）；EVYD Technology, Singapore（EVYD技术公司，新加坡）
Multi-Layer Personalized Federated Learning for Mitigating Biases in Student Predictive Analytics
Elmore Family School of Electrical and Computer Engineering, Purdue University（普渡大学埃尔莫尔家族电气与计算机工程学院）；University at Buffalo (SUNY)（纽约州立大学布法罗分校）；University of Florida（佛罗里达大学）；School of Engineering Education, Purdue University（普渡大学工程教育学院）；Manning College of Information and Computer Sciences, University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校曼宁信息与计算机科学学院）
Mitigating Malicious Attacks in Federated Learning via Confidence-aware Defense
Queen Mary University of London（伦敦玛丽女王大学）；University of Oxford（牛津大学）；Huazhong University of Science and Technology（华中科技大学）；Shandong University of Technology（山东理工大学）
National Key Laboratory of Novel Software Technology（计算机软件新技术国家重点实验室）；Nanjing University（南京大学）
In reinforcement learning, experience replay stores past samples for further reuse. Prioritized sampling is a promising technique to better utilize these samples. Previous criteria of prioritization include TD error, recentness and corrective feedback, which are mostly heuristically designed. In this work, we start from the regret minimization objective, and obtain an optimal prioritization strategy for Bellman update that can directly maximize the return of the policy. The theory suggests that data with higher hindsight TD error, better on-policiness and more accurate Q value should be assign…
Federated and distributed learning applications for electronic health records and structured medical data: A scoping review
Duke-NUS Medical School（杜克-新加坡国立大学医学院）；National Dental Research Institute Singapore（新加坡国家牙科研究所）；National Dental Centre Singapore（新加坡国家牙科中心）；Oral Health Academic Clinical Programme, Duke-NUS Medical School（杜克-新加坡国立大学医学院口腔健康学术临床项目）；Programme in Health Services and Systems Research, Duke-NUS Medical School（杜克-新加坡国立大学医学院卫生服务与系统研究项目）；National University of Singapore（新加坡国立大学）；Duke University（杜克大学）；Singapore National Eye Centre（新加坡国家眼科中心）；Singapore Eye Research Institute（新加坡眼科研究所）；Imperial College London（伦敦帝国理工学院）；Singapore General Hospital（新加坡中央医院）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究院）
Federated Averaging (FedAvg) and its variants are the most popular optimization algorithms in federated learning (FL). Previous convergence analyses of FedAvg either assume full client participation or partial client participation where the clients can be uniformly sampled. However, in practical cross-device FL systems, only a subset of clients that satisfy local criteria such as battery status, network connectivity, and maximum participation frequency requirements (to ensure privacy) are available for training at a given time. As a result, client availability follows a natural cyclic pattern.…
Nowadays, gathering high-quality training data from multiple data sources with privacy preservation is a crucial challenge to training high-performance machine learning models. The potential solutions could break the barriers among isolated data corpus, and consequently enlarge the range of data available for processing. To this end, both academic researchers and industrial vendors are recently strongly motivated to propose two main-stream folders of solutions mainly based on software constructions: 1) Secure Multi-party Learning (MPL for short); and 2) Federated Learning (FL for short). The a…
