Smartphones, autonomous vehicles, and the Internet-of-things (IoT) devices are considered the primary data source for a distributed network. Due to a revolutionary breakthrough in internet availability and continuous improvement of the IoT devices capabilities, it is desirable to store data locally and perform computation at the edge, as opposed to share all local information with a centralized computation agent. A recently proposed Machine Learning (ML) algorithm called Federated Learning (FL) paves the path towards preserving data privacy, performing distributed learning, and reducing communication overhead in large-scale machine learning (ML) problems. This paper proposes an FL model by monitoring client activities and leveraging available local computing resources, particularly for resource-constrained IoT devices (e.g., mobile robots), to accelerate the learning process. We assign a trust score to each FL client, which is updated based on the client's activities. We consider a distributed mobile robot as an FL client with resource limitations either in memory, bandwidth, processor, or battery life. We consider such mobile robots as FL clients to understand their resource-constrained behavior in a real-world setting. We consider an FL client to be untrustworthy if the client infuses incorrect models or repeatedly gives slow responses during the FL process. After disregarding the ineffective and unreliable client, we perform local training on the selected FL clients. To further reduce the straggler issue, we enable an asynchronous FL mechanism by performing aggregation on the FL server without waiting for a long period to receive a particular client's response.
Indian Institute of Technology (BHU)（印度理工学院（贝拿勒斯印度教大学））；Missouri University of Science and Technology（密苏里科技大学）
With the enhancement of people's living standards and rapid growth of communication technologies, residential environments are becoming smart and well-connected, increasing overall energy consumption substantially. As household appliances are the primary energy consumers, their recognition becomes crucial to avoid unattended usage, thereby conserving energy and making smart environments more sustainable. An appliance recognition model is traditionally trained at a central server (service provider) by collecting electricity consumption data, recorded via smart plugs, from the clients (consumers…
Global AI Accelerator, Ericsson（爱立信全球人工智能加速器）；Ericsson Research（爱立信研究院）
Various health-care applications such as assisted living, fall detection, etc., require modeling of user behavior through Human Activity Recognition (HAR). Such applications demand characterization of insights from multiple resource-constrained user devices using machine learning techniques for effective personalized activity monitoring. On-device Federated Learning proves to be an effective approach for distributed and collaborative machine learning. However, there are a variety of challenges in addressing statistical (non-IID data) and model heterogeneities across users. In addition, in this…
We show that a word-level recurrent neural network can predict emoji from text typed on a mobile keyboard. We demonstrate the usefulness of transfer learning for predicting emoji by pretraining the model using a language modeling task. We also propose mechanisms to trigger emoji and tune the diversity of candidates. The model is trained using a distributed on-device learning framework called federated learning. The federated model is shown to achieve better performance than a server-trained model. This work demonstrates the feasibility of using federated learning to train production-quality mo…
Federated learning is used for decentralized training of machine learning models on a large number (millions) of edge mobile devices. It is challenging because mobile devices often have limited communication bandwidth and local computation resources. Therefore, improving the efficiency of federated learning is critical for scalability and usability. In this paper, we propose to leverage partially trainable neural networks, which freeze a portion of the model parameters during the entire training process, to reduce the communication cost with little implications on model performance. Through ex…
Nanyang Technological University（南洋理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Sungkyunkwan University（成均馆大学）
To strengthen data privacy and security, federated learning as an emerging machine learning technique is proposed to enable large-scale nodes, e.g., mobile devices, to distributedly train and globally share models without revealing their local data. This technique can not only significantly improve privacy protection for mobile devices, but also ensure good performance of the trained results collectively. Currently, most the existing studies focus on optimizing federated learning algorithms to improve model training performance. However, incentive mechanisms to motivate the mobile devices to j…
The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Federated learning (FL) enables collaborative model training without centralizing data. However, the traditional FL framework is cloud-based and suffers from high communication latency. On the other hand, the edge-based FL framework that relies on an edge server co-located with mobile base station for model aggregation has low communication latency but suffers from degraded model accuracy due to the limited coverage of edge server. In light of high accuracy but high-latency cloud-based FL and low-latency but low-accuracy edge-based FL, this paper proposes a new FL framework based on cooperativ…
University of Houston（休斯顿大学）；Xidian University（西安电子科技大学）；University of Florida（佛罗里达大学）
The continuous convergence of machine learning algorithms, 5G and beyond (5G+) wireless communications, and artificial intelligence (AI) hardware implementation hastens the birth of federated learning (FL) over 5G+ mobile devices, which pushes AI functions to mobile devices and initiates a new era of on-device AI applications. Despite the remarkable progress made in FL, huge energy consumption is one of the most significant obstacles restricting the development of FL over battery-constrained 5G+ mobile devices. To address this issue, in this paper, we investigate how to develop energy efficien…
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Cyberspace Security Research Center, Peng Cheng Laboratory（鹏城实验室网络空间安全研究中心）；School of Electrical and Data Engineering, University of Technology Sydney（悉尼科技大学电气与数据工程学院）；Department of Computer Science and Technology, Fuzhou University（福州大学计算机科学与技术学院）；Department of Computer Science, Electrical and Space Engineering, Lulea University of Technology（吕勒奥理工大学计算机、电气与空间工程系）
In recent years, mobile clients' computing ability and storage capacity have greatly improved, efficiently dealing with some applications locally. Federated learning is a promising distributed machine learning solution that uses local computing and local data to train the Artificial Intelligence (AI) model. Combining local computing and federated learning can train a powerful AI model under the premise of ensuring local data privacy while making full use of mobile clients' resources. However, the heterogeneity of local data, that is, Non-independent and identical distribution (Non-IID) and imb…
Xidian University（西安电子科技大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Houston（休斯顿大学）
Recent advances in machine learning, wireless communication, and mobile hardware technologies promisingly enable federated learning (FL) over massive mobile edge devices, which opens new horizons for numerous intelligent mobile applications. Despite the potential benefits, FL imposes huge communication and computation burdens on participating devices due to periodical global synchronization and continuous local training, raising great challenges to battery constrained mobile devices. In this work, we target at improving the energy efficiency of FL over mobile edge networks to accommodate heter…
Kyung Hee University（庆熙大学）；The University of Sydney（悉尼大学）；University of Houston（休斯顿大学）
Federated Learning is a new learning scheme for collaborative training a shared prediction model while keeping data locally on participating devices. In this paper, we study a new model of multiple federated learning services at the multi-access edge computing server. Accordingly, the sharing of CPU resources among learning services at each mobile device for the local training process and allocating communication resources among mobile devices for exchanging learning information must be considered. Furthermore, the convergence performance of different learning services depends on the hyper-lea…
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Mathematics and Computer Science, Fuzhou University（福州大学数学与计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Mobile crowdsensing (MCS) is an emerging sensing data collection pattern with scalability, low deployment cost, and distributed characteristics. Traditional MCS systems suffer from privacy concerns and fair reward distribution. Moreover, existing privacy-preserving MCS solutions usually focus on the privacy protection of data collection rather than that of data processing. To tackle faced problems of MCS, in this paper, we integrate federated learning (FL) into MCS and propose a privacy-preserving MCS system, called \textsc{CrowdFL}. Specifically, in order to protect privacy, participants loca…
University of Houston（休斯顿大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；University of Science and Technology of China（中国科学技术大学）；University of Florida（佛罗里达大学）
Federated learning (FL) is a popular collaborative distributed machine learning paradigm across mobile devices. However, practical FL over resource constrained mobile devices confronts multiple challenges, e.g., the local on-device training and model updates in FL are power hungry and radio resource intensive for mobile devices. To address these challenges, in this paper, we attempt to take FL into the design of future wireless networks and develop a novel joint design of wireless transmission and weight quantization for energy efficient FL over mobile devices. Specifically, we develop flexibl…
Singapore University of Design and Technology（新加坡科技设计大学）；East China Normal University（华东师范大学）；University of Electronic Science and Technology of China（电子科技大学）；Beihang University（北京航空航天大学）
Federated Learning (FL) can be used in mobile edge networks to train machine learning models in a distributed manner. Recently, FL has been interpreted within a Model-Agnostic Meta-Learning (MAML) framework, which brings FL significant advantages in fast adaptation and convergence over heterogeneous datasets. However, existing research simply combines MAML and FL without explicitly addressing how much benefit MAML brings to FL and how to maximize such benefit over mobile edge networks. In this paper, we quantify the benefit from two aspects: optimizing FL hyperparameters (i.e., sampled data si…
Singapore University of Design and Technology（新加坡科技设计大学）；Shenzhen University（深圳大学）；East China Normal University（华东师范大学）；Zhejiang University（浙江大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Personalized Federated Learning (PFL) is a new Federated Learning (FL) approach to address the heterogeneity issue of the datasets generated by distributed user equipments (UEs). However, most existing PFL implementations rely on synchronous training to ensure good convergence performances, which may lead to a serious straggler problem, where the training time is heavily prolonged by the slowest UE. To address this issue, we propose a semi-synchronous PFL algorithm, termed as Semi-Synchronous Personalized FederatedAveraging (PerFedS$^2$), over mobile edge networks. By jointly optimizing the wi…
Ericsson Research（爱立信研究院）；Ericsson Security Research（爱立信安全研究院）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Machine Learning (ML) is an important enabler for optimizing, securing and managing mobile networks. This leads to increased collection and processing of data from network functions, which in turn may increase threats to sensitive end-user information. Consequently, mechanisms to reduce threats to end-user privacy are needed to take full advantage of ML. We seamlessly integrate Federated Learning (FL) into the 3GPP 5G Network Data Analytics (NWDA) architecture, and add a Multi-Party Computation (MPC) protocol for protecting the confidentiality of local updates. We evaluate the protocol and fin…
Federated Learning in Mobile Networks: A Comprehensive Case Study on Traffic Forecasting
Democritus University of Thrace（德谟克利特色雷斯大学）；Imperial College London（帝国理工学院）；Centre Tecnològic de Telecomunicacions de Catalunya (CTTC/CERCA)（加泰罗尼亚电信技术中心（CTTC/CERCA））；Nokia Bell Labs（诺基亚贝尔实验室）；IMT Atlantique（大西洋电信学校）；Inria（法国国家信息与自动化研究所）；LS2N（南特计算机科学与实验室（注：此处LS2N为实验室，按要求保留独立机构名））
In order to improve mobile data transparency, a number of network-based approaches have been proposed to inspect packets generated by mobile devices and detect personally identifiable information (PII), ad requests, or other activities. State-of-the-art approaches train classifiers based on features extracted from HTTP packets. So far, these classifiers have only been trained in a centralized way, where mobile users label and upload their packet logs to a central server, which then trains a global classifier and shares it with the users to apply on their devices. However, packet logs used as t…
Federated Learning (FL) enables statistical models to be built on user-generated data without compromising data security and user privacy. For this reason, FL is well suited for on-device learning from mobile devices where data is abundant and highly privatized. Constrained by the temporal availability of mobile devices, only a subset of devices is accessible to participate in the iterative protocol consisting of training and aggregation. In this study, we take a step toward better understanding the effect of non-independent data distributions arising from block-cyclic sampling. By conducting …
Tianjin University（天津大学）；Huawei Technology（华为技术）；Sun Yat-sen University（中山大学）；Huazhong University of Science and Technology（华中科技大学）
Recently, along with the rapid development of mobile communication technology, edge computing theory and techniques have been attracting more and more attentions from global researchers and engineers, which can significantly bridge the capacity of cloud and requirement of devices by the network edges, and thus can accelerate the content deliveries and improve the quality of mobile services. In order to bring more intelligence to the edge systems, compared to traditional optimization methodology, and driven by the current deep learning techniques, we propose to integrate the Deep Reinforcement …
REWAFL: Residual Energy and Wireless Aware Participant Selection for Efficient Federated Learning over Mobile Devices
Beijing University of Posts and Telecommunications（北京邮电大学）；Peng Cheng Laboratory（鹏城实验室）；University of Houston（休斯顿大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Federated learning (FL) has emerged as an effective solution to decentralized and privacy-preserving machine learning for mobile clients. While traditional FL has demonstrated its superiority, it ignores the non-iid (independently identically distributed) situation, which widely exists in mobile scenarios. Failing to handle non-iid situations could cause problems such as performance decreasing and possible attacks. Previous studies focus on the "symptoms" directly, as they try to improve the accuracy or detect possible attacks by adding extra steps to conventional FL models. However, previous …
Owing to the large volume of sensed data from the enormous number of IoT devices in operation today, centralized machine learning algorithms operating on such data incur an unbearable training time, and thus cannot satisfy the requirements of delay-sensitive inference applications. By provisioning computing resources at the network edge, Mobile Edge Computing (MEC) has become a promising technology capable of collaborating with distributed IoT devices to facilitate federated learning, and thus realize real-time training. However, considering the large volume of sensed data and the limited reso…
Macquarie University（麦考瑞大学）；University of New South Wales（新南威尔士大学）
Federated Learning (FL) is an efficient distributed machine learning paradigm that employs private datasets in a privacy-preserving manner. The main challenges of FL is that end devices usually possess various computation and communication capabilities and their training data are not independent and identically distributed (non-IID). Due to limited communication bandwidth and unstable availability of such devices in a mobile network, only a fraction of end devices (also referred to as the participants or clients in a FL process) can be selected in each round. Hence, it is of paramount importan…
North China Electric Power University（华北电力大学）；Fayetteville State University（费耶特维尔州立大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Qatar University（卡塔尔大学）
The concept of federated learning (FL) was first proposed by Google in 2016. Thereafter, FL has been widely studied for the feasibility of application in various fields due to its potential to make full use of data without compromising the privacy. However, limited by the capacity of wireless data transmission, the employment of federated learning on mobile devices has been making slow progress in practical. The development and commercialization of the 5th generation (5G) mobile networks has shed some light on this. In this paper, we analyze the challenges of existing federated learning scheme…
FedMood: Federated Learning on Mobile Health Data for Mood Detection
School of Cyber Science and Technology, Beihang University（北京航空航天大学网络空间科学与技术学院）；Beijing Advanced Innovation Center for Big Data and Brain Computing（北京大数据与脑计算高精尖创新中心）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Fordham University（福特汉姆大学）；Lehigh University（理海大学）
Federated learning (FL) is a machine learning technique that aims at training an algorithm across decentralized entities holding their local data private. Wireless mobile networks allow users to communicate with other fixed or mobile users. The road traffic network represents an infrastructure-based configuration of a wireless mobile network where the Connected and Automated Vehicles (CAV) represent the communicating entities. Applying FL in a wireless mobile network setting gives rise to a new threat in the mobile environment that is very different from the traditional fixed networks. The thr…
Being very low power, the use of neuromorphic processors in mobile devices to solve machine learning problems is a promising alternative to traditional Von Neumann processors. Federated Learning enables entities such as mobile devices to collaboratively learn a shared model while keeping their training data local. Additionally, federated learning is a secure way of learning because only the model weights need to be shared between models, keeping the data private. Here we demonstrate the efficacy of federated learning in neuromorphic processors. Neuromorphic processors benefit from the collabor…
University of Houston（休斯顿大学）；Guangdong University of Technology（广东工业大学）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））
Federated learning (FL), an emerging distributed machine learning paradigm, in conflux with edge computing is a promising area with novel applications over mobile edge devices. In FL, since mobile devices collaborate to train a model based on their own data under the coordination of a central server by sharing just the model updates, training data is maintained private. However, without the central availability of data, computing nodes need to communicate the model updates often to attain convergence. Hence, the local computation time to create local model updates along with the time taken for…
Mobile Edge Computing (MEC) has been a promising paradigm for communicating and edge processing of data on the move. We aim to employ Federated Learning (FL) and prominent features of blockchain into MEC architecture such as connected autonomous vehicles to enable complete decentralization, immutability, and rewarding mechanisms simultaneously. FL is advantageous for mobile devices with constrained connectivity since it requires model updates to be delivered to a central point instead of substantial amounts of data communication. For instance, FL in autonomous, connected vehicles can increase …
University of Houston（休斯顿大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Purple Mountain Laboratories（紫金山实验室）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
Federated learning (FL) over mobile devices has fostered numerous intriguing applications/services, many of which are delay-sensitive. In this paper, we propose a service delay efficient FL (SDEFL) scheme over mobile devices. Unlike traditional communication efficient FL, which regards wireless communications as the bottleneck, we find that under many situations, the local computing delay is comparable to the communication delay during the FL training process, given the development of high-speed wireless transmission techniques. Thus, the service delay in FL should be computing delay + communi…
Beijing University of Posts and Telecommunications（北京邮电大学）；University of Houston（休斯顿大学）；Louisiana State University（路易斯安那州立大学）；Shandong Artificial Intelligence Institute（山东省人工智能研究院）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））
Federated learning (FL) enables mobile devices to collaboratively learn a shared prediction model while keeping data locally. However, there are two major research challenges to practically deploy FL over mobile devices: (i) frequent wireless updates of huge size gradients v.s. limited spectrum resources, and (ii) energy-hungry FL communication and local computing during training v.s. battery-constrained mobile devices. To address those challenges, in this paper, we propose a novel multi-bit over-the-air computation (M-AirComp) approach for spectrum-efficient aggregation of local model updates…
FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selection
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学）；College of Electronic Information Engineering, Taiyuan University of Technology（电子信息工程学院，太原科技大学）；Department of Advanced Interdisciplinary Research, Pengcheng Laboratory（先进跨学科研究部，鹏城实验室）；Department of Electrical and Computer Engineering, University of Houston（电气与计算机工程系，德克萨斯大学）
Singapore University of Design and Technology（新加坡科技设计大学）；East China Normal University（华东师范大学）；Zhejiang University（浙江大学）
Personalized Federated Learning (PFL) is a new Federated Learning (FL) paradigm, particularly tackling the heterogeneity issues brought by various mobile user equipments (UEs) in mobile edge computing (MEC) networks. However, due to the ever-increasing number of UEs and the complicated administrative work it brings, it is desirable to switch the PFL algorithm from its conventional two-layer framework to a multiple-layer one. In this paper, we propose hierarchical PFL (HPFL), an algorithm for deploying PFL over massive MEC networks. The UEs in HPFL are divided into multiple clusters, and the UE…
New Jersey Institute of Technology（新泽西理工学院）；Qualcomm Incorporated（高通公司）；Kent State University（肯特州立大学）
Mobile apps, such as mHealth and wellness applications, can benefit from deep learning (DL) models trained with mobile sensing data collected by smart phones or wearable devices. However, currently there is no mobile sensing DL system that simultaneously achieves good model accuracy while adapting to user mobility behavior, scales well as the number of users increases, and protects user data privacy. We propose Zone-based Federated Learning (ZoneFL) to address these requirements. ZoneFL divides the physical space into geographical zones mapped to a mobile-edge-cloud system architecture for goo…
Privacy-Preserving Edge Federated Learning for Intelligent Mobile-Health Systems
Institute of Computer Engineering, Heidelberg University（海德堡大学计算机工程研究所）；Faculty of Computer Engineering, K. N. Toosi University of Technology（霍加·纳赛尔·图西大学计算机工程学院）；Department of Electrical and Information Technology, Lund University（隆德大学电气与信息技术系）
Federated learning (FL) aided health diagnostic models can incorporate data from a large number of personal edge devices (e.g., mobile phones) while keeping the data local to the originating devices, largely ensuring privacy. However, such a cross-device FL approach for health diagnostics still imposes many challenges due to both local data imbalance (as extreme as local data consists of a single disease class) and global data imbalance (the disease prevalence is generally low in a population). Since the federated server has no access to data distribution information, it is not trivial to solv…
University of Technology Sydney（悉尼科技大学）；Universitas Gadjah Mada（加查马达大学）；Pusan National University（釜山国立大学）
In this work, we propose a novel framework to address straggling and privacy issues for federated learning (FL)-based mobile application services, taking into account limited computing/communications resources at mobile users (MUs)/mobile application provider (MAP), privacy cost, the rationality and incentive competition among MUs in contributing data to the MAP. Particularly, the MAP first determines a set of the best MUs for the FL process based on the MUs' provided information/features. To mitigate straggling problems with privacy-awareness, each selected MU can then encrypt part of local d…
Federated Learning-Assisted Optimization of Mobile Transmission with Digital Twins
Department of Electrical and Computer Engineering, McMaster University, CANADA（电气与计算机工程系，麦斯特大学）；Department of Computing and Software, McMaster University, CANADA（计算与软件系，麦斯特大学）
Chongqing University（重庆大学）；Beijing Institute of Control Engineering（北京控制工程研究所）
Federated learning (FL) is a distributed deep learning method which enables multiple participants, such as mobile phones and IoT devices, to contribute a neural network model while their private training data remains in local devices. This distributed approach is promising in the edge computing system where have a large corpus of decentralized data and require high privacy. However, unlike the common training dataset, the data distribution of the edge computing system is imbalanced which will introduce biases in the model training and cause a decrease in accuracy of federated learning applicat…
North Carolina State University（北卡罗来纳州立大学）；Wuhan University（武汉大学）
In federated learning (FL), reducing the communication overhead is one of the most critical challenges since the parameter server and the mobile devices share the training parameters over wireless links. With such consideration, we adopt the idea of SignSGD in which only the signs of the gradients are exchanged. Moreover, most of the existing works assume Channel State Information (CSI) available at both the mobile devices and the parameter server, and thus the mobile devices can adopt fixed transmission rates dictated by the channel capacity. In this work, only the parameter server side CSI i…
Graduate School of Informatics, Kyoto University（京都大学信息学研究科）；Institute of Industrial Science, The University of Tokyo（东京大学工业科学研究所）
We envision a mobile edge computing (MEC) framework for machine learning (ML) technologies, which leverages distributed client data and computation resources for training high-performance ML models while preserving client privacy. Toward this future goal, this work aims to extend Federated Learning (FL), a decentralized learning framework that enables privacy-preserving training of models, to work with heterogeneous clients in a practical cellular network. The FL protocol iteratively asks random clients to download a trainable model from a server, update it with own data, and upload the update…
