Internet of Things (IoT) with its growing number of deployed devices and applications raises significant challenges for network maintenance procedures. In this work, we formulate a problem of autonomous maintenance in IoT networks as a Partially Observable Markov Decision Process. Subsequently, we utilize Deep Reinforcement Learning algorithms (DRL) to train agents that decide if a maintenance procedure is in order or not and, in the former case, the proper type of maintenance needed. To avoid wasting the scarce resources of IoT networks we utilize the Age of Information (AoI) metric as a reward signal for the training of the smart agents. AoI captures the freshness of the sensory data which are transmitted by the IoT sensors as part of their normal service provision. Numerical results indicate that AoI integrates enough information about the past and present states of the system to be successfully used in the training of smart agents for the autonomous maintenance of the network.
AoI-Aware Task Offloading and Transmission Optimization for Industrial IoT Networks: A Branching Deep Reinforcement Learning Approach
Laboratory of Future Networks, University of Science and Technology of China（未来网络实验室，中国科学技术大学）；Deep Space Exploration Laboratory（深空探测实验室）；Department of Computing, The Hong Kong Polytechnic University（computing系，香港理工大学）
ICT Research Institute（信息通信技术研究所）；Tarbiat Modares University（塔尔比亚特莫达雷斯大学）；University of Ottawa（渥太华大学）
In delay-sensitive industrial internet of things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such pro…
Beijing University of Posts and Telecommunications（北京邮电大学）；Princeton University（普林斯顿大学）；Virginia Tech（弗吉尼亚理工大学）；Shenzhen Research Institute of Big Data (SRIBD)（深圳市大数据研究院）；Future Network of Intelligence Institute (FNii), Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）未来智联网络研究院）
In this paper, the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices is studied. In the considered model, each IoT device monitors a physical process that follows nonlinear dynamics. As the dynamics of the physical process vary over time, each device must find an optimal sampling frequency to sample the real-time dynamics of the physical system and send sampled information to a base station (BS). Due to limited wireless resources, the BS can only select a subset of devices to transmit their sampled information. T…
Age of Information (AoI) has gained importance as a Key Performance Indicator (KPI) for characterizing the freshness of information in information-update systems and time-critical applications. Recent theoretical research on the topic has generated significant understanding of how various algorithms perform in terms of this metric on various system models and networking scenarios. In this paper, by the help of the theoretical results, we analyzed the AoI behavior on real-life networks, using our two test-beds, addressing IoT networks and regular computers. Excessive number of AoI measurements …
Age and Power Minimization via Meta-Deep Reinforcement Learning in UAV Networks
Centre for Wireless Communications (CWC), University of Oulu（奥卢大学无线通信中心）；CEMSE Division, King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学CEMSE学部）
George Mason University（乔治梅森大学）；Virginia Tech（弗吉尼亚理工学院暨州立大学）
Due to flexibility, autonomy and low operational cost, unmanned aerial vehicles (UAVs), as fixed aerial base stations, are increasingly being used as \textit{relays} to collect time-sensitive information (i.e., status updates) from IoT devices and deliver it to the nearby terrestrial base station (TBS), where the information gets processed. In order to ensure timely delivery of information to the TBS (from all IoT devices), optimal scheduling of time-sensitive information over two hop UAV-relayed IoT networks (i.e., IoT device to the UAV [hop 1], and UAV to the TBS [hop 2]) becomes a critical …
As the most well-known application of the Internet of Things (IoT), remote monitoring is now pervasive. In these monitoring applications, information usually has a higher value when it is fresher. A new metric, termed the age of information (AoI), has recently been proposed to quantify the information freshness in various IoT applications. This paper concentrates on the design and analysis of age-oriented random access for massive IoT networks. Specifically, we devise a new stationary threshold-based age-dependent random access (ADRA) protocol, in which each IoT device accesses the channel wit…
Centre for Wireless Communications, University of Oulu（奥卢大学无线通信中心）
While information delivery in industrial Internet of things demands reliability and latency guarantees, the freshness of the controller's available information, measured by the age of information (AoI), is paramount for high-performing industrial automation. The problem in this work is cast as a sensor's transmit power minimization subject to the peak-AoI requirement and a probabilistic constraint on queuing latency. We further characterize the tail behavior of the latency by a generalized Pareto distribution (GPD) for solving the power allocation problem through Lyapunov optimization. As each…
Caching has been regarded as a promising technique to alleviate energy consumption of sensors in Internet of Things (IoT) networks by responding to users' requests with the data packets stored in the edge caching node (ECN). For real-time applications in caching enabled IoT networks, it is essential to develop dynamic status update strategies to strike a balance between the information freshness experienced by users and energy consumed by the sensor, which, however, is not well addressed. In this paper, we first depict the evolution of information freshness, in terms of age of information (AoI…
We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor. The edge node serves users' requests by deciding whether to command the corresponding sensor to send a fresh status update or retrieve the most recently received measurement from the cache. Our objective is to find the best actions of the edge node to minimize the average age of information (AoI) of the received measurements upon request, i.e., average on-demand AoI, subjec…
In this paper, we consider the multicast transmission of a real-time Internet of Things (IoT) system, where an access point (AP) transmits time-stamped status updates to multiple IoT devices. Different from the existing studies that only considered multicast transmission without deadlines, we enforce a deadline for the service time of each multicast status update, taking into account both the fixed and randomly distributed deadlines. In particular, a status update is dropped when either its deadline expires or it is successfully received by a certain number of IoT devices. Considering deadline…
In this article, we provide an accessible introduction to the emerging idea of Age of Information (AoI) that quantifies freshness of information and explore its possible role in the efficient design of freshness-aware Internet of Things (IoT). We start by summarizing the concept of AoI and its variants with emphasis on the differences between AoI and other well-known performance metrics in the literature, such as throughput and delay. Building on this, we explore freshness-aware IoT design for a network in which IoT devices sense potentially different physical processes and are supposed to fre…
The Fusion of Deep Reinforcement Learning and Edge Computing for Real-time Monitoring and Control Optimization in IoT Environments
Northern Arizona University（北亚利桑那大学）；University of Electronic Science and Technology of China（电子科技大学）；Trine University（特莱恩大学）；Yantai University（烟台大学）；Northwestern University（西北大学）
This work studies a real-time environment monitoring scenario in the industrial Internet of things, where wireless sensors proactively collect environmental data and transmit it to the controller. We adopt the notion of risk-sensitivity in financial mathematics as the objective to jointly minimize the mean, variance, and other higher-order statistics of the network energy consumption subject to the constraints on the age of information (AoI) threshold violation probability and the AoI exceedances over a pre-defined threshold. We characterize the extreme AoI staleness using results in extreme v…
This paper considers a cognitive radio-based IoT monitoring system, consisting of an IoT device that aims to update its measurement to a destination using cognitive radio technique. Specifically, the IoT device as a secondary user (SIoT), seeks and exploits the spectrum opportunities of the licensed band vacated by its primary user (PU) to deliver status updates without causing visible effects to the licensed operation. In this context, the SIoT should carefully make use of the licensed band and schedule when to transmit to maintain the timeliness of the status update. We adopt a recent metric…
The widespread adoption of 5G cellular technology will evolve as one of the major drivers for the growth of IoT-based applications. In this paper, we consider a Service Provider (SP) that launches a smart city service based on IoT data readings: in order to serve IoT data collected across different locations, the SP dynamically negotiates and rescales bandwidth and service functions. 5G network slicing functions are key to lease an appropriate amount of resources over heterogeneous access technologies and different site types. Also, different infrastructure providers will charge slicing servic…
This paper considers a cooperative Internet of Things (IoT) system with a source aiming to transmit randomly generated status updates to a designated destination as timely as possible under the help of a relay. We adopt a recently proposed concept, the age of information (AoI), to characterize the timeliness of the status updates. In the considered system, delivering the status updates via the one-hop direct link will have a shorter transmission time at the cost of incurring a higher error probability, while the delivery of status updates through the two-hop relay link could be more reliable a…
This paper highlights the trends in the field of predictive maintenance with the use of machine learning. With the continuous development of the Fourth Industrial Revolution, through IoT, the technologies that use artificial intelligence are evolving. As a result, industries have been using these technologies to optimize their production. Through scientific research conducted for this paper, conclusions were drawn about the trends in Predictive Maintenance applications with the use of machine learning bridging Artificial Intelligence and IoT. These trends are related to the types of industries…
In this paper, a dense Internet of Things (IoT) monitoring system is studied in which a large number of devices contend for transmitting timely status packets to their corresponding receivers over wireless noisy channels, using a carrier sense multiple access (CSMA) scheme. When each device completes one transmission, due to possible transmission failure, two cases with and without transmission feedback to each device must be considered. Particularly, for the case with no feedback, the device uses policy (I): It will go to an idle state and release the channel regardless of the outcome of the …
Minimization of the expected value of age of information (AoI) is a risk-neutral approach, and it thus cannot capture rare, yet critical, events with potentially large AoI. In order to capture the effect of these events, in this paper, the notion of conditional value-at-risk (CVaR) is proposed as an effective coherent risk measure that is suitable for minimization of AoI for real-time IoT status updates. In the considered monitoring system, an IoT device monitors a physical process and sends the status updates to a remote receiver with an updating cost. The optimal status update process is des…
More and more emerging Internet of Things (IoT) applications involve status updates, where various IoT devices monitor certain physical processes and report their latest statuses to the relevant information fusion nodes. A new performance measure, termed the age of information (AoI), has recently been proposed to quantify the information freshness in time-critical IoT applications. Due to a large number of devices in future IoT networks, the decentralized channel access protocols (e.g. random access) are preferable thanks to their low network overhead. Built on the AoI concept, some recent eff…
Due to the flexibility and low operational cost, dispatching unmanned aerial vehicles (UAVs) to collect information from distributed sensors is expected to be a promising solution in Internet of Things (IoT), especially for time-critical applications. How to maintain the information freshness is a challenging issue. In this paper, we investigate the fresh data collection problem in UAV-assisted IoT networks. Particularly, the UAV flies towards the sensors to collect status update packets within a given duration while maintaining a non-negative residual energy. We formulate a Markov Decision Pr…
Age of information (AoI) is gaining attention as a valuable performance metric for many IoT systems, in which a large number of devices report time-stamped updates to a central gateway. This is the case, for instance, of remote sensing, monitoring, or tracking, with broad applications in the industrial, vehicular, and environmental domain. In these settings, AoI provides insights that are complementary to those offered by throughput or latency, capturing the ability of the system to maintain an up-to-date view of the status of each transmitting device. From this standpoint, while a good unders…
In many emerging Internet of Things (IoT) applications, the freshness of the is an important design criterion. Age of Information (AoI) quantifies the freshness of the received information or status update. This work considers a setup of deployed IoT devices in an IoT network; multiple unmanned aerial vehicles (UAVs) serve as mobile relay nodes between the sensors and the base station. We formulate an optimization problem to jointly plan the UAVs' trajectory, while minimizing the AoI of the received messages and the devices' energy consumption. The solution accounts for the UAVs' battery lifet…
In the Internet of Things (IoT) networks, caching is a promising technique to alleviate energy consumption of sensors by responding to users' data requests with the data packets cached in the edge caching node (ECN). However, without an efficient status update strategy, the information obtained by users may be stale, which in return would inevitably deteriorate the accuracy and reliability of derived decisions for real-time applications. In this paper, we focus on striking the balance between the information freshness, in terms of age of information (AoI), experienced by users and energy consu…
Timeliness is an emerging requirement for many Internet of Things (IoT) applications. In IoT networks, where a large-number of nodes are distributed, severe interference may incur during the transmission phase which causes age of information (AoI) degradation. It is therefore important to study the performance limit of AoI as well as how to achieve such limit. In this paper, we aim to optimize the AoI in random access Poisson networks. By taking into account the spatio-temporal interactions amongst the transmitters, an expression of the peak AoI is derived, based on explicit expressions of the…
Scheduling updates from remote sensors is a key task for the internet of things (IoT). In this context, the mathematical concept of age of information is often used to capture the freshness of received data. This is, in turn, relevant to optimize the frequency of the exchanges, especially for resource constrained (e.g., energy-limited) sensors. Most investigations on the subject assume that the transmitter can leverage knowledge of the age of information at the receiver side to decide when to send data, even when the communication channel is unreliable. In reality, tracking the outcome of the …
In this paper, a real-time Internet of Things (IoT) monitoring system is considered in which the IoT devices are scheduled to sample underlying physical processes and send the status updates to a common destination. In a real-world IoT, due to the possibly different dynamics of each physical process, the sizes of the status updates for different devices are often different and each status update typically requires multiple transmission slots. By taking into account such multi-time slot transmissions with non-uniform sizes of the status updates under noisy channels, the problem of joint device …
The freshness of status information is of great importance for time-critical Internet of Things (IoT) applications. A metric measuring status freshness is the age-of-information (AoI), which captures the time elapsed from the status being generated at the source node (e.g., a sensor) to the latest status update.However, in intelligent IoT applications such as video surveillance, the status information is revealed after some computation intensive and time-consuming data processing operations, which would affect the status freshness. In this paper, we propose a novel metric, age-of-processing (A…
The present paper focuses on an IoT setting in which a large number of devices generate time-stamped updates addressed to a common gateway. Medium access is regulated following a grant-free approach, and the system aims at maintaining an up-to-date knowledge at the receiver, measured through the average network age of information (AoI). In this context, we propose a variation of the irregular repetition slotted ALOHA (IRSA) protocol. The scheme, referred to as age-threshold IRSA (AT-IRSA), leverages feedback provided by the receiver to admit to the channel only devices whose AoI exceeds a dyna…
Low Earth Orbit (LEO) satellite constellations are bringing the Internet of Things (IoT) to the space arena, also known as non-terrestrial networks. Several IoT satellite applications for tracking ships and cargo can be seen as exemplary cases of intermittent transmission of updates whose main performance parameter is the information freshness. This paper analyzes the Age of Information (AoI) of a satellite network with multiple sources and destinations that are very distant and therefore require several consecutive multi-hop transmissions. A packet erasure channel and different queueing polic…
The Internet of Things (IoT) has been applied to a large number of heterogeneous devices and is used in the deployment of a variety of applications on the basis of its distributed open architecture. The majority of these IoT devices are battery-powered and are interconnected via a wireless network. IoT devices may be used to carry out critical tasks. Thus, the IoT network requires a resilient architecture that supports semantic search, failure discovery, data recovery, and dynamic and autonomous network maintenance. In this paper, we present a new resilience scheme for IoT networks. We evaluat…
