In this paper, we examine cloud-edge-terminal IoT networks, where edges undertake a range of typical dynamic scheduling tasks. In these IoT networks, a central policy for each task can be constructed at a cloud server. The central policy can be then used by the edges conducting the task, thereby mitigating the need for them to learn their own policy from scratch. Furthermore, this central policy can be collaboratively learned at the cloud server by aggregating local experiences from the edges, thanks to the hierarchical architecture of the IoT networks. To this end, we propose a novel collaborative policy learning framework for dynamic scheduling tasks using federated reinforcement learning. For effective learning, our framework adaptively selects the tasks for collaborative learning in each round, taking into account the need for fairness among tasks. In addition, as a key enabler of the framework, we propose an edge-agnostic policy structure that enables the aggregation of local policies from different edges. We then provide the convergence analysis of the framework. Through simulations, we demonstrate that our proposed framework significantly outperforms the approaches without collaborative policy learning. Notably, it accelerates the learning speed of the policies and allows newly arrived edges to adapt to their tasks more easily.
University of Waterloo（滑铁卢大学）；Zhejiang University（浙江大学）；Huazhong University of Science and Technology（华中科技大学）；Central South University（中南大学）；Xidian University（西安电子科技大学）
In this paper, we investigate a computing task scheduling problem in space-air-ground integrated network (SAGIN) for delay-oriented Internet of Things (IoT) services. In the considered scenario, an unmanned aerial vehicle (UAV) collects computing tasks from IoT devices and then makes online offloading decisions, in which the tasks can be processed at the UAV or offloaded to the nearby base station or the remote satellite. Our objective is to design a task scheduling policy that minimizes offloading and computing delay of all tasks given the UAV energy capacity constraint. To this end, we first…
Q-Learning-Based Time-Critical Data Aggregation Scheduling in IoT
Convergence Research Institute, Sungkyunkwan University, Suwon, South Korea（顺天大学 convergence 研究院，韩国首尔南）；School of Electrical and Electronic Engineering, Hanoi University of Science and Technology, Hanoi, Vietnam（河内科学技术大学电气与电子工程学院，越南河内）；Dept. of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, South Korea（顺天大学电气与计算机工程系，韩国首尔南）
Addressing Terminal Constraints in Data-Driven Demand Response Scheduling
Sargent Centre for Process Systems Engineering, Imperial College London, London, SW7 2AZ, UK（过程系统工程中心，伦敦帝国理工学院，伦敦，SW7 2AZ，英国）；Department of Computer Science, University of Alberta, Edmonton, AB, Canada（计算机科学系，阿尔伯塔大学，埃德蒙顿，AB，加拿大）；Department of Computing, Imperial College London, London, SW7 2AZ, UK（计算系，伦敦帝国理工学院，伦敦，SW7 2AZ，英国）
ICT Research Institute（信息通信技术研究所）；Tarbiat Modares University（塔尔比亚特莫达雷斯大学）；University of Ottawa（渥太华大学）
In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such pro…
The University of Melbourne（墨尔本大学）；Indian Institute of Technology, Delhi（印度理工学院德里分校）
The ubiquitous adoption of Internet-of-Things (IoT) based applications has resulted in the emergence of the Fog computing paradigm, which allows seamlessly harnessing both mobile-edge and cloud resources. Efficient scheduling of application tasks in such environments is challenging due to constrained resource capabilities, mobility factors in IoT, resource heterogeneity, network hierarchy, and stochastic behaviors. xisting heuristics and Reinforcement Learning based approaches lack generalizability and quick adaptability, thus failing to tackle this problem optimally. They are also unable to u…
This paper proposes a Time Division Multiple Access (TDMA) MAC slot allocation protocol with efficient bandwidth usage in wireless sensor networks and Internet of Things (IoTs). The developed protocol has two primary components: a Multi-Armed Bandits (MAB)-based slot allocation mechanism for collision free transmission, and a Decentralized Defragmented Slot Backshift (DDSB) operation for improving bandwidth usage efficiency. The proposed framework is decentralized in that each node finds its transmission schedule independently without the control of any centralized arbitrator. The developed me…
Queen’s University Belfast（贝尔法斯特女王大学）；HoChiMinh City University of Technology and Education（胡志明市工业与教育大学）；Princeton University（普林斯顿大学）
Many of the devices used in Internet-of-Things (IoT) applications are energy-limited, and thus supplying energy while maintaining seamless connectivity for IoT devices is of considerable importance. In this context, we propose a simultaneous wireless power transfer and information transmission scheme for IoT devices with support from reconfigurable intelligent surface (RIS)-aided unmanned aerial vehicle (UAV) communications. In particular, in a first phase, IoT devices harvest energy from the UAV through wireless power transfer; and then in a second phase, the UAV collects data from the IoT de…
University of Ottawa（渥太华大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Aerial base stations (ABSs) allow smart farms to offload processing responsibility of complex tasks from internet of things (IoT) devices to ABSs. IoT devices have limited energy and computing resources, thus it is required to provide an advanced solution for a system that requires the support of ABSs. This paper introduces a novel multi-actor-based risk-sensitive reinforcement learning approach for ABS task scheduling for smart agriculture. The problem is defined as task offloading with a strict condition on completing the IoT tasks before their deadlines. Moreover, the algorithm must also co…
The Fusion of Deep Reinforcement Learning and Edge Computing for Real-time Monitoring and Control Optimization in IoT Environments
Northern Arizona University（北亚利桑那大学）；University of Electronic Science and Technology of China（电子科技大学）；Trine University（特莱恩大学）；Yantai University（烟台大学）；Northwestern University（西北大学）
State-Aware IoT Scheduling Using Deep Q-Networks and Edge-Based Coordination
New York University（纽约大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；Vanderbilt University（范德堡大学）；Northeastern University（东北大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Royal Institute of Technology (KTH)（皇家理工学院（KTH））；Princeton University（普林斯顿大学）
To leverage massive distributed data and computation resources, machine learning in the network edge is considered to be a promising technique especially for large-scale model training. Federated learning (FL), as a paradigm of collaborative learning techniques, has obtained increasing research attention with the benefits of communication efficiency and improved data privacy. Due to the lossy communication channels and limited communication resources (e.g., bandwidth and power), it is of interest to investigate fast responding and accurate FL schemes over wireless systems. Hence, we investigat…
We investigate resource allocation scheme to reduce the energy consumption of federated learning (FL) in the integrated fog-cloud computing enabled Internet-of-things (IoT) networks. In the envisioned system, IoT devices are connected with the centralized cloud server (CS) via multiple fog access points (F-APs). We consider two different scenarios for training the local models. In the first scenario, local models are trained at the IoT devices and the F-APs upload the local model parameters to the CS. In the second scenario, local models are trained at the F-APs based on the collected data fro…
Resource Efficient Asynchronous Federated Learning for Digital Twin Empowered IoT Network
School of Electronic and Optical Engineering, Nanjing University of Science and Technology（南京理工大学电子工程与光电技术学院）；School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；Jiangsu Second Normal University（江苏第二师范学院）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）；Department of Electronic Engineering, Shanghai Jiao Tong University（上海交通大学电子工程系）；National Mobile Communication Research Laboratory, Southeast University（东南大学国家移动通信研究实验室）
Agency for Science, Technology and Research（新加坡科技研究局）；Nanyang Technological University（南洋理工大学）
Given the trend of digitization and increasing number of maritime transport, prediction of vessel berth stay has been triggered for requirements of operation research and scheduling optimization problem in the era of maritime big data, which takes a significant part in port efficiency and maritime logistics enhancement. This study proposes a systematic and dynamic approach of predicting berth stay for tanker terminals. The approach covers three innovative aspects: 1) Data source employed is multi-faceted, including cargo operation data from tanker terminals, time-series data from automatic ide…
Robust Generalization of Graph Neural Networks for Carrier Scheduling
RISE Computer Science（RISE计算机科学研究院）；KTH Royal Institute of Technology（皇家理工学院）；Digital Futures（数字未来研究院）；Karolinska Institutet（卡罗林斯卡学院）；MedTechLabs（医疗技术实验室）；Uppsala University（乌普萨拉大学）
