ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, Canada（电气与计算机工程系，不列颠哥伦比亚大学）；Department of Computer Science and Engineering, Southern University of Science and Technology, China（计算机科学与工程系，南方科技大学）
Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively.
Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks…
HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning
University of Naples "Federico II"（那不勒斯腓特烈二世大学）；Queen Mary University of London（伦敦玛丽女王大学）；Brandenburg University of Technology Cottbus-Senftenberg（勃兰登堡工业大学）；Tallinn University of Technology（塔林理工大学）；University of Zanjan（赞詹大学）
This paper studies a federated learning (FL) system, where \textit{multiple} FL services co-exist in a wireless network and share common wireless resources. It fills the void of wireless resource allocation for multiple simultaneous FL services in the existing literature. Our method designs a two-level resource allocation framework comprising \emph{intra-service} resource allocation and \emph{inter-service} resource allocation. The intra-service resource allocation problem aims to minimize the length of FL rounds by optimizing the bandwidth allocation among the clients of each FL service. Base…
University of Oulu（奥卢大学）；Centre for Wireless Communications（无线通信中心）
The performance of federated learning (FL) over wireless networks depend on the reliability of the client-server connectivity and clients' local computation capabilities. In this article we investigate the problem of client scheduling and resource block (RB) allocation to enhance the performance of model training using FL, over a pre-defined training duration under imperfect channel state information (CSI) and limited local computing resources. First, we analytically derive the gap between the training losses of FL with clients scheduling and a centralized training method for a given training …
This paper studies federated learning (FL) in a classic wireless network, where learning clients share a common wireless link to a coordinating server to perform federated model training using their local data. In such wireless federated learning networks (WFLNs), optimizing the learning performance depends crucially on how clients are selected and how bandwidth is allocated among the selected clients in every learning round, as both radio and client energy resources are limited. While existing works have made some attempts to allocate the limited wireless resources to optimize FL, they focus …
Small-Scale-Fading-Aware Resource Allocation in Wireless Federated Learning
National Mobile Communications Research Laboratory, Southeast University（东南大学国家移动通信研究中心）；Purple Mountain Laboratories（紫金山实验室）；Department of Electrical and Computer Engineering, University of California, Santa Cruz（加州大学圣克鲁兹分校电气与计算机工程系）；College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）
Beijing University of Posts and Telecommunications（北京邮电大学）；Princeton University（普林斯顿大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Limited radio frequency (RF) resources restrict the number of users that can participate in federated learning (FL) thus affecting FL convergence speed and performance. In this paper, we first introduce visible light communication (VLC) as a supplement to RF in FL and build a hybrid VLC/RF communication system, in which each indoor user can use both VLC and RF to transmit its FL model parameters. Then, the problem of user selection and bandwidth allocation is studied for FL implemented over a hybrid VLC/RF system aiming to optimize the FL performance. The problem is first separated into two su…
University of Electronic Science and Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）
This work poses a distributed multi-resource allocation scheme for minimizing the weighted sum of latency and energy consumption in the on-device distributed federated learning (FL) system. Each mobile device in the system engages the model training process within the specified area and allocates its computation and communication resources for deriving and uploading parameters, respectively, to minimize the objective of system subject to the computation/communication budget and a target latency requirement. In particular, mobile devices are connect via wireless TCP/IP architectures. Exploiting…
Sejong University（世宗大学）；Pusan National University（釜山国立大学）
Recently, a considerable amount of works have been made to tackle the communication burden in federated learning (FL) (e.g., model quantization, data sparsification, and model compression). However, the existing methods, that boost the communication efficiency in FL, result in a considerable trade-off between communication efficiency and global convergence rate. We formulate an optimization problem for compression-aided FL, which captures the relationship between the distortion rate, number of participating IoT devices, and convergence rate. Following that, the objective function is to minimiz…
Hitachi (China) Research & Development Cooperation（日立（中国）研发有限公司）
In federated learning (FL), devices contribute to the global training by uploading their local model updates via wireless channels. Due to limited computation and communication resources, device scheduling is crucial to the convergence rate of FL. In this paper, we propose a joint device scheduling and resource allocation policy to maximize the model accuracy within a given total training time budget for latency constrained wireless FL. A lower bound on the reciprocal of the training performance loss, in terms of the number of training rounds and the number of scheduled devices per round, is d…
Utah State University（犹他州立大学）；University of Wisconsin-Whitewater（威斯康星大学怀特沃特分校）
Federated learning (FL) is a highly pursued machine learning technique that can train a model centrally while keeping data distributed. Distributed computation makes FL attractive for bandwidth limited applications especially in wireless communications. There can be a large number of distributed edge devices connected to a central parameter server (PS) and iteratively download/upload data from/to the PS. Due to the limited bandwidth, only a subset of connected devices can be scheduled in each round. There are usually millions of parameters in the state-of-art machine learning models such as de…
BGTplanner: Maximizing Training Accuracy for Differentially Private Federated Recommenders via Strategic Privacy Budget Allocation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）；Department of Computing, Faculty of Science and Engineering, Macquarie University（麦考瑞大学计算机系）；Machine Learning Department, Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学机器学习系）
Convergence-Latency-Aware Adaptive Modulation and Resource Allocation in RIS-Assisted Wireless Federated Learning
Broadband Access Network Laboratory, Shanghai Jiao Tong University（宽带接入网络实验室，上海交通大学）；School of Information Science and Engineering, Southeast University（信息科学与工程学院，东南大学）；Data61, CSIRO（数据61，联邦科学与工业研究组织）；Department of Electronic and Electrical Engineering, University College London（电子与电气工程系，伦敦大学学院）；School of Internet of Things Engineering, Jiangnan University（物联网工程学院，江南大学）
Swiss Re（瑞士再保险）；Ping An P&C Insurance Co., Ltd.（平安财产保险股份有限公司）
Federated Machine Learning (FML) creates an ecosystem for multiple parties to collaborate on building models while protecting data privacy for the participants. A measure of the contribution for each party in FML enables fair credits allocation. In this paper we develop simple but powerful techniques to fairly calculate the contributions of multiple parties in FML, in the context of both horizontal FML and vertical FML. For Horizontal FML we use deletion method to calculate the grouped instance influence. For Vertical FML we use Shapley Values to calculate the grouped feature importance. Our m…
University of Virginia（弗吉尼亚大学）；University of Miami（迈阿密大学）；Sun Yat-Sen University（中山大学）
We advocate a new resource allocation framework, which we term resource rationing, for wireless federated learning (FL). Unlike existing resource allocation methods for FL, resource rationing focuses on balancing resources across learning rounds so that their collective impact on the federated learning performance is explicitly captured. This new framework can be integrated seamlessly with existing resource allocation schemes to optimize the convergence of FL. In particular, a novel "later-is-better" principle is at the front and center of resource rationing, which is validated empirically in …
Industrial Technology Research Institute（工业技术研究院）；National Yang Ming Chiao Tung University（阳明交通大学）
Federated learning aims to share private data to maximize the data utility without privacy leakage. Previous federated learning research mainly focuses on multi-class classification problems. However, multi-label classification is a crucial research problem close to real-world data properties. Nevertheless, a limited number of federated learning studies explore this research problem. Existing studies of multi-label federated learning did not consider the characteristics of multi-label data, i.e., they used the concept of multi-class classification to verify their methods' performance, which me…
Nanjing University of Science and Technology（南京理工大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；University of Houston（休斯顿大学）；Princeton University（普林斯顿大学）
Federated learning (FL), as a distributed machine learning paradigm, promotes personal privacy by local data processing at each client. However, relying on a centralized server for model aggregation, standard FL is vulnerable to server malfunctions, untrustworthy server, and external attacks. To address this issue, we propose a decentralized FL framework by integrating blockchain into FL, namely, blockchain assisted decentralized federated learning (BLADE-FL). In a round of the proposed BLADE-FL, each client broadcasts the trained model to other clients, aggregates its own model with received …
Indiana University-Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）；Bowling Green State University（鲍灵格林州立大学）；Shandong University（山东大学）
Blockchain-based federated learning (BCFL) has recently gained tremendous attention because of its advantages such as decentralization and privacy protection of raw data. However, there has been few research focusing on the allocation of resources for clients in BCFL. In the BCFL framework where the FL clients and the blockchain miners are the same devices, clients broadcast the trained model updates to the blockchain network and then perform mining to generate new blocks. Since each client has a limited amount of computing resources, the problem of allocating computing resources into training…
George Mason University（乔治梅森大学）；Microsoft（微软）；University of Maryland, Baltimore County（马里兰大学巴尔的摩分校）
Federated learning learns from scattered data by fusing collaborative models from local nodes. However, the conventional coordinate-based model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. Fed2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation…
We investigate resource allocation scheme to reduce the energy consumption of federated learning (FL) in the integrated fog-cloud computing enabled Internet-of-things (IoT) networks. In the envisioned system, IoT devices are connected with the centralized cloud server (CS) via multiple fog access points (F-APs). We consider two different scenarios for training the local models. In the first scenario, local models are trained at the IoT devices and the F-APs upload the local model parameters to the CS. In the second scenario, local models are trained at the F-APs based on the collected data fro…
Virginia Tech（弗吉尼亚理工大学）；University of Colorado Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）；Ericsson Research（爱立信研究院）；Princeton University（普林斯顿大学）；University of Oulu（奥卢大学）
Unmanned aerial vehicle (UAV) swarms must exploit machine learning (ML) in order to execute various tasks ranging from coordinated trajectory planning to cooperative target recognition. However, due to the lack of continuous connections between the UAV swarm and ground base stations (BSs), using centralized ML will be challenging, particularly when dealing with a large volume of data. In this paper, a novel framework is proposed to implement distributed federated learning (FL) algorithms within a UAV swarm that consists of a leading UAV and several following UAVs. Each following UAV trains a l…
In this paper, we study a new latency optimization problem for blockchain-based federated learning (BFL) in multi-server edge computing. In this system model, distributed mobile devices (MDs) communicate with a set of edge servers (ESs) to handle both machine learning (ML) model training and block mining simultaneously. To assist the ML model training for resource-constrained MDs, we develop an offloading strategy that enables MDs to transmit their data to one of the associated ESs. We then propose a new decentralized ML model aggregation solution at the edge layer based on a consensus mechani…
Latency Minimization for UAV-Enabled Federated Learning: Trajectory Design and Resource Allocation
Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)（深圳未来智能网络研究院）；School of Science and Engineering (SSE)（科学与工程学院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Institute of Advanced Technology (SIAT)（深圳先进技术研究院）；Chinese Academy of Sciences (CAS)（中国科学院）；Southern University of Science and Technology（南方科技大学）；Imperial College London（伦敦帝国学院）；Central South University（中南大学）；Shenzhen University of Advanced Technology（深圳理工大学）
Beijing University of Posts and Telecommunications（北京邮电大学）；Princeton University（普林斯顿大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Virginia Tech（弗吉尼亚理工大学）；Kyung Hee University（庆熙大学）
In this paper, the problem of minimizing energy and time consumption for task computation and transmission is studied in a mobile edge computing (MEC)-enabled balloon network. In the considered network, each user needs to process a computational task in each time instant, where high-altitude balloons (HABs), acting as flying wireless base stations, can use their powerful computational abilities to process the tasks offloaded from their associated users. Since the data size of each user's computational task varies over time, the HABs must dynamically adjust the user association, service sequenc…
Singapore University of Design and Technology（新加坡科技设计大学）；Shenzhen University（深圳大学）；East China Normal University（华东师范大学）；Zhejiang University（浙江大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Personalized Federated Learning (PFL) is a new Federated Learning (FL) approach to address the heterogeneity issue of the datasets generated by distributed user equipments (UEs). However, most existing PFL implementations rely on synchronous training to ensure good convergence performances, which may lead to a serious straggler problem, where the training time is heavily prolonged by the slowest UE. To address this issue, we propose a semi-synchronous PFL algorithm, termed as Semi-Synchronous Personalized FederatedAveraging (PerFedS$^2$), over mobile edge networks. By jointly optimizing the wi…
Beijing University of Posts and Telecommunications（北京邮电大学）；North China Electric Power University（华北电力大学）；Commonwealth Science and Industrial Research Organisation（联邦科学与工业研究组织）
Wireless federated learning (WFL) undergoes a communication bottleneck in uplink, limiting the number of users that can upload their local models in each global aggregation round. This paper presents a new multi-carrier non-orthogonal multiple-access (MC-NOMA)-empowered WFL system under an adaptive learning setting of Flexible Aggregation. Since a WFL round accommodates both local model training and uploading for each user, the use of Flexible Aggregation allows the users to train different numbers of iterations per round, adapting to their channel conditions and computing resources. The key i…
In this paper we envision a federated learning (FL) scenario in service of amending the performance of autonomous road vehicles, through a drone traffic monitor (DTM), that also acts as an orchestrator. Expecting non-IID data distribution, we focus on the issue of accelerating the learning of a particular class of critical object (CO), that may harm the nominal operation of an autonomous vehicle. This can be done through proper allocation of the wireless resources for addressing learner and data heterogeneity. Thus, we propose a reactive method for the allocation of wireless resources, that ha…
Inclusive Federated Learning Through Compliance-Weighted Noise Allocation in Healthcare AI
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Heidelberg University（海德堡大学）；Bilkent University（比尔肯特大学）；Indiana University（印第安纳大学）；Heidelberg University Hospital（海德堡大学医院）；MLCommons
Department of Electrical and Computer Engineering, Princeton University（普林斯顿大学电气与计算机工程系）；Department of Engineering, King’s College London（伦敦国王学院工程系）；Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications（北京邮电大学网络系统架构与融合北京市重点实验室）；Shenzhen Research Institute of Big Data and Future Network of Intelligence Institute, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）智能科学与技术研究院未来网络与大数据研究院深圳分院）
In this paper, the problem of training federated learning (FL) algorithms over a realistic wireless network is studied. In particular, in the considered model, wireless users execute an FL algorithm while training their local FL models using their own data and transmitting the trained local FL models to a base station (BS) that will generate a global FL model and send it back to the users. Since all training parameters are transmitted over wireless links, the quality of the training will be affected by wireless factors such as packet errors and the availability of wireless resources. Meanwhile…
Royal Institute of Technology (KTH)（皇家理工学院（KTH））；Princeton University（普林斯顿大学）
To leverage massive distributed data and computation resources, machine learning in the network edge is considered to be a promising technique especially for large-scale model training. Federated learning (FL), as a paradigm of collaborative learning techniques, has obtained increasing research attention with the benefits of communication efficiency and improved data privacy. Due to the lossy communication channels and limited communication resources (e.g., bandwidth and power), it is of interest to investigate fast responding and accurate FL schemes over wireless systems. Hence, we investigat…
Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity
Shanghai Jiao Tong University（上海交通大学）；Nanjing University of Science and Technology（南京理工大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）；Princeton University（普林斯顿大学）
