This paper studies an edge intelligence-based IoT network in which a set of edge servers learn a shared model using federated learning (FL) based on the datasets uploaded from a multi-technology-supported IoT network. The data uploading performance of IoT network and the computational capacity of edge servers are entangled with each other in influencing the FL model training process. We propose a novel framework, called federated edge intelligence (FEI), that allows edge servers to evaluate the required number of data samples according to the energy cost of the IoT network as well as their local data processing capacity and only request the amount of data that is sufficient for training a satisfactory model. We evaluate the energy cost for data uploading when two widely-used IoT solutions: licensed band IoT (e.g., 5G NB-IoT) and unlicensed band IoT (e.g., Wi-Fi, ZigBee, and 5G NR-U) are available to each IoT device. We prove that the cost minimization problem of the entire IoT network is separable and can be divided into a set of subproblems, each of which can be solved by an individual edge server. We also introduce a mapping function to quantify the computational load of edge servers under different combinations of three key parameters: size of the dataset, local batch size, and number of local training passes. Finally, we adopt an Alternative Direction Method of Multipliers (ADMM)-based approach to jointly optimize energy cost of the IoT network and average computing resource utilization of edge servers. We prove that our proposed algorithm does not cause any data leakage nor disclose any topological information of the IoT network. Simulation results show that our proposed framework significantly improves the resource efficiency of the IoT network and edge servers with only a limited sacrifice on the model convergence performance.
Internet of Things (IoT) have widely penetrated in different aspects of modern life and many intelligent IoT services and applications are emerging. Recently, federated learning is proposed to train a globally shared model by exploiting a massive amount of user-generated data samples on IoT devices while preventing data leakage. However, the device, statistical and model heterogeneities inherent in the complex IoT environments pose great challenges to traditional federated learning, making it unsuitable to be directly deployed. In this article we advocate a personalized federated learning fram…
Beijing University of Posts and Telecommunications（北京邮电大学）
Implementing existing federated learning in massive Internet of Things (IoT) networks faces critical challenges such as imbalanced and statistically heterogeneous data and device diversity. To this end, we propose a semi-federated learning (SemiFL) framework to provide a potential solution for the realization of intelligent IoT. By seamlessly integrating the centralized and federated paradigms, our SemiFL framework shows high scalability in terms of the number of IoT devices even in the presence of computing-limited sensors. Furthermore, compared to traditional learning approaches, the propose…
Pusan National University（釜山大学）；University of Technology Sydney（悉尼科技大学）
Federated learning (FL) is a new artificial intelligence concept that enables Internet-of-Things (IoT) devices to learn a collaborative model without sending the raw data to centralized nodes for processing. Despite numerous advantages, low computing resources at IoT devices and high communication costs for exchanging model parameters make applications of FL in massive IoT networks very limited. In this work, we develop a novel compression scheme for FL, called high-compression federated learning (HCFL), for very large scale IoT networks. HCFL can reduce the data load for FL processes without …
Rice University（莱斯大学）；University of Houston-Downtown（休斯顿大学市中心分校）；Prudence Medical Technologies Ltd.（审慎医疗技术有限公司）
The Internet of Things (IoT) revolution has shown potential to give rise to many medical applications with access to large volumes of healthcare data collected by IoT devices. However, the increasing demand for healthcare data privacy and security makes each IoT device an isolated island of data. Further, the limited computation and communication capacity of wearable healthcare devices restrict the application of vanilla federated learning. To this end, we propose an advanced federated learning framework to train deep neural networks, where the network is partitioned and allocated to IoT devic…
University of Calgary（卡尔加里大学）；Michigan State University（密歇根州立大学）
Federated learning (FL) is a distributed and privacy-preserving learning framework for predictive modeling with massive data generated at the edge by Internet of Things (IoT) devices. One major challenge preventing the wide adoption of FL in IoT is the pervasive power supply constraints of IoT devices due to the intensive energy consumption of battery-powered clients for local training and model updates. Low battery levels of clients eventually lead to their early dropouts from edge networks, loss of training data jeopardizing the performance of FL, and their availability to perform other desi…
Pennsylvania State University（宾夕法尼亚州立大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Purdue University（普渡大学）；Georgia State University（佐治亚州立大学）
Federated learning (FL) enables multiple clients to train models collaboratively without sharing local data, which has achieved promising results in different areas, including the Internet of Things (IoT). However, end IoT devices do not have abilities to automatically annotate their collected data, which leads to the label shortage issue at the client side. To collaboratively train an FL model, we can only use a small number of labeled data stored on the server. This is a new yet practical scenario in federated learning, i.e., labels-at-server semi-supervised federated learning (SemiFL). Alth…
Federated Learning (FL) has received a significant amount of attention in the industry and research community due to its capability of keeping data on local devices. To aggregate the gradients of local models to train the global model, existing works require that the global model and the local models are the same. However, Internet of Things (IoT) devices are inherently diverse regarding computation speed and onboard memory. In this paper, we propose an FL framework targeting the heterogeneity of IoT devices. Specifically, local models are compressed from the global model, and the gradients of…
Recent years have witnessed a rapid proliferation of smart Internet of Things (IoT) devices. IoT devices with intelligence require the use of effective machine learning paradigms. Federated learning can be a promising solution for enabling IoT-based smart applications. In this paper, we present the primary design aspects for enabling federated learning at network edge. We model the incentive-based interaction between a global server and participating devices for federated learning via a Stackelberg game to motivate the participation of the devices in the federated learning process. We present …
Lebanese American University（黎巴嫩美国大学）；Polytechnique Montréal（蒙特利尔理工学院）；Khalifa University（哈利法大学）；Zayed University（扎耶德大学）；New York University（纽约大学）；Mohammad Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Federated Learning (FL) is a novel distributed privacy-preserving learning paradigm, which enables the collaboration among several participants (e.g., Internet of Things devices) for the training of machine learning models. However, selecting the participants that would contribute to this collaborative training is highly challenging. Adopting a random selection strategy would entail substantial problems due to the heterogeneity in terms of data quality, and computational and communication resources across the participants. Although several approaches have been proposed in the literature to ove…
The ongoing deployment of the Internet of Things (IoT)-based smart applications is spurring the adoption of machine learning as a key technology enabler. To overcome the privacy and overhead challenges of centralized machine learning, there has been a significant recent interest in the concept of federated learning. Federated learning offers on-device, privacy-preserving machine learning without the need to transfer end-devices data to a third party location. However, federated learning still has privacy concerns due to sensitive information inferring capability of the aggregation server using…
University of Pittsburgh（匹兹堡大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Internet of Things (IoT) devices are becoming increasingly popular and are influencing many application domains such as healthcare and transportation. These devices are used for real-world applications such as sensor monitoring, real-time control. In this work, we look at differentially private (DP) neural network (NN) based network intrusion detection systems (NIDS) to detect intrusion attacks on networks of such IoT devices. Existing NN training solutions in this domain either ignore privacy considerations or assume that the privacy requirements are homogeneous across all users. We show that…
École de Technologie Supérieure (ÉTS)（高等技术学院（ÉTS））；Universite de Quebec a Trois-Rivieres (UQTR)（魁北克大学三河分校（UQTR））；Lebanese American University（黎巴嫩美国大学）；New York University（纽约大学）；Mohammad Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Continuous behavioural authentication methods add a unique layer of security by allowing individuals to verify their unique identity when accessing a device. Maintaining session authenticity is now feasible by monitoring users' behaviour while interacting with a mobile or Internet of Things (IoT) device, making credential theft and session hijacking ineffective. Such a technique is made possible by integrating the power of artificial intelligence and Machine Learning (ML). Most of the literature focuses on training machine learning for the user by transmitting their data to an external server,…
Constrained Bayesian Optimization for Hierarchical Federated Learning in IoT Networks for Plant Disease Classification
University of Zagreb（萨格勒布大学）；University of Thessaly（塞萨利大学）；International Hellenic University（国际希腊大学）；University of Glasgow（格拉斯哥大学）；Trinity College Dublin（都柏林圣三一学院）；Italian National Research Council（意大利国家研究委员会）
MicroService Architecture (MSA) is gaining rapid popularity for developing large-scale IoT applications for deployment within distributed and resource-constrained Fog computing environments. As a cloud-native application architecture, the true power of microservices comes from their loosely coupled, independently deployable and scalable nature, enabling distributed placement and dynamic composition across federated Fog and Cloud clusters. Thus, it is necessary to develop novel microservice placement algorithms that utilise these microservice characteristics to improve the performance of the ap…
National Institute of Posts and Telecommunications（国家邮电学院）
Federated Machine Learning (Fed ML) is a new distributed machine learning technique applied to collaboratively train a global model using clients local data without transmitting it. Nodes only send parameter updates (e.g., weight updates in the case of neural networks), which are fused together by the server to build the global model. By not divulging node data, Fed ML guarantees its confidentiality, a crucial aspect of network security, which enables it to be used in the context of data-sensitive Internet of Things (IoT) and mobile applications, such as smart Geo-location and the smart grid. …
Northwestern Polytechnical University（西北工业大学）；Xidian University（西安电子科技大学）；University of Oslo（奥斯陆大学）；Simula Metropolitan Center for Digital Engineering（西穆拉都会数字工程中心）
Industrial Internet of Things (IoT) enables distributed intelligent services varying with the dynamic and realtime industrial devices to achieve Industry 4.0 benefits. In this paper, we consider a new architecture of digital twin empowered Industrial IoT where digital twins capture the characteristics of industrial devices to assist federated learning. Noticing that digital twins may bring estimation deviations from the actual value of device state, a trusted based aggregation is proposed in federated learning to alleviate the effects of such deviation. We adaptively adjust the aggregation fre…
Royal Institute of Technology (KTH)（皇家理工学院（KTH））；Princeton University（普林斯顿大学）
To leverage massive distributed data and computation resources, machine learning in the network edge is considered to be a promising technique especially for large-scale model training. Federated learning (FL), as a paradigm of collaborative learning techniques, has obtained increasing research attention with the benefits of communication efficiency and improved data privacy. Due to the lossy communication channels and limited communication resources (e.g., bandwidth and power), it is of interest to investigate fast responding and accurate FL schemes over wireless systems. Hence, we investigat…
Recently, IoT technologies have been progressed, and many sensors and actuators are connected to networks. Previously, IoT services were developed by vertical integration style. But now Open IoT concept has attracted attentions which achieves various IoT services by integrating horizontal separated devices and services. For Open IoT era, we have proposed the Tacit Computing technology to discover the devices with necessary data for users on demand and use them dynamically. We also implemented elemental technologies of Tacit Computing. In this paper, we propose three layers optimizations to red…
Pennsylvania State University（宾夕法尼亚州立大学）；Fudan University（复旦大学）
Federated learning (FL) and split learning (SL) are two emerging collaborative learning methods that may greatly facilitate ubiquitous intelligence in Internet of Things (IoT). Federated learning enables machine learning (ML) models locally trained using private data to be aggregated into a global model. Split learning allows different portions of an ML model to be collaboratively trained on different workers in a learning framework. Federated learning and split learning, each has unique advantages and respective limitations, may complement each other toward ubiquitous intelligence in IoT. The…
Resource Efficient Asynchronous Federated Learning for Digital Twin Empowered IoT Network
School of Electronic and Optical Engineering, Nanjing University of Science and Technology（南京理工大学电子工程与光电技术学院）；School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；Jiangsu Second Normal University（江苏第二师范学院）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）；Department of Electronic Engineering, Shanghai Jiao Tong University（上海交通大学电子工程系）；National Mobile Communication Research Laboratory, Southeast University（东南大学国家移动通信研究实验室）
Device failure detection is one of most essential problems in industrial internet of things (IIoT). However, in conventional IIoT device failure detection, client devices need to upload raw data to the central server for model training, which might lead to disclosure of sensitive business data. Therefore, in this paper, to ensure client data privacy, we propose a blockchain-based federated learning approach for device failure detection in IIoT. First, we present a platform architecture of blockchain-based federated learning systems for failure detection in IIoT, which enables verifiable integr…
Hamad Bin Khalifa University（哈马德·本·哈利法大学）；American University of Beirut（贝鲁特美国大学）；Qatar University（卡塔尔大学）
Federated learning (FL) is a distributed learning methodology that allows multiple nodes to cooperatively train a deep learning model, without the need to share their local data. It is a promising solution for telemonitoring systems that demand intensive data collection, for detection, classification, and prediction of future events, from different locations while maintaining a strict privacy constraint. Due to privacy concerns and critical communication bottlenecks, it can become impractical to send the FL updated models to a centralized server. Thus, this paper studies the potential of hiera…
Western Illinois University（西伊利诺伊大学）；Western Michigan University（西密歇根大学）；Al-Nahrain University（纳赫拉因大学）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）；Mohammed V University（穆罕默德五世大学）；Information Technology University（信息技术大学）；University of Nebraska at Kearney（内布拉斯加大学卡尼尔分校）；Qatar University（卡塔尔大学）
Machine Learning (ML), and Deep Learning (DL) in particular, play a vital role in providing smart services to the industry. These techniques however suffer from privacy and security concerns since data is collected from clients and then stored and processed at a central location. Federated Learning (FL), an architecture in which model parameters are exchanged instead of client data, has been proposed as a solution to these concerns. Nevertheless, FL trains a global model by communicating with clients over communication rounds, which introduces more traffic on the network and increases the conv…
University of St Andrews（圣安德鲁斯大学）；Queen’s University Belfast（贝尔法斯特女王大学）；Rakuten Mobile（乐天移动）
Applying Federated Learning (FL) on Internet-of-Things devices is necessitated by the large volumes of data they produce and growing concerns of data privacy. However, there are three challenges that need to be addressed to make FL efficient: (i) execution on devices with limited computational capabilities, (ii) accounting for stragglers due to computational heterogeneity of devices, and (iii) adaptation to the changing network bandwidths. This paper presents FedAdapt, an adaptive offloading FL framework to mitigate the aforementioned challenges. FedAdapt accelerates local training in computat…
Florida International University（佛罗里达国际大学）；Arm ML Research（Arm机器学习研究中心）；IBM T. J. Watson Research Center（IBM托马斯·J·沃森研究中心）；Binghamton University, the State University of New York（纽约州立大学宾汉姆顿分校）
Nowadays, devices are equipped with advanced sensors with higher processing/computing capabilities. Further, widespread Internet availability enables communication among sensing devices. As a result, vast amounts of data are generated on edge devices to drive Internet-of-Things (IoT), crowdsourcing, and other emerging technologies. The collected extensive data can be pre-processed, scaled, classified, and finally, used for predicting future events using machine learning (ML) methods. In traditional ML approaches, data is sent to and processed in a central server, which encounters communication…
Heilongjiang University（黑龙江大学）；École de technologie supérieure, Université du Québec（魁北克大学高等技术学院）；Nanyang Technological University（南洋理工大学）；Wuhan University of Technology（武汉理工大学）；Alibaba-NTU Joint Research Institute（阿里巴巴-南洋理工大学联合研究院）
Since edge device failures (i.e., anomalies) seriously affect the production of industrial products in Industrial IoT (IIoT), accurately and timely detecting anomalies is becoming increasingly important. Furthermore, data collected by the edge device may contain the user's private data, which is challenging the current detection approaches as user privacy is calling for the public concern in recent years. With this focus, this paper proposes a new communication-efficient on-device federated learning (FL)-based deep anomaly detection framework for sensing time-series data in IIoT. Specifically,…
Consiglio Nazionale delle Ricerche（意大利国家研究委员会）；Insitute of Electronics, Computer and Telecommunication Engineering（电子、计算机与电信工程研究所）；Politecnico di Milano（米兰理工大学）
Federated learning (FL) is emerging as a new paradigm to train machine learning models in distributed systems. Rather than sharing, and disclosing, the training dataset with the server, the model parameters (e.g. neural networks weights and biases) are optimized collectively by large populations of interconnected devices, acting as local learners. FL can be applied to power-constrained IoT devices with slow and sporadic connections. In addition, it does not need data to be exported to third parties, preserving privacy. Despite these benefits, a main limit of existing approaches is the centrali…
A significant bottleneck in federated learning (FL) is the network communication cost of sending model updates from client devices to the central server. We present a comprehensive empirical study of the statistics of model updates in FL, as well as the role and benefits of various compression techniques. Motivated by these observations, we propose a novel method to reduce the average communication cost, which is near-optimal in many use cases, and outperforms Top-K, DRIVE, 3LC and QSGD on Stack Overflow next-word prediction, a realistic and challenging FL benchmark. This is achieved by examin…
