Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle
School of Robot Engineering, Yangtze Normal University（长江师范学院机器人工程学院）；College of Automotive Engineering, Chongqing University（重庆大学汽车工程学院）；School of intelligent manufacturing engineering, Chongqing University of Arts and Sciences（重庆文理学院智能制造工程学院）；Department of Mechanical and Mechatronics Engineering, University of Waterloo（滑铁卢大学机械与机电工程学院）
Xiaowei Guo, Teng Liu, Bangbei Tang, Xiaolin Tang, Jinwei Zhang, Wenhao Tan, Shufeng Jin
This paper proposes an adaptive energy management strategy for hybrid electric vehicles by combining deep reinforcement learning (DRL) and transfer learning (TL). This work aims to address the defect of DRL in tedious training time. First, an optimization control modeling of a hybrid tracked vehicle is built, wherein the elaborate powertrain components are introduced. Then, a bi-level control framework is constructed to derive the energy management strategies (EMSs). The upper-level is applying the particular deep deterministic policy gradient (DDPG) algorithms for EMS training at different speed intervals. The lower-level is employing the TL method to transform the pre-trained neural networks for a novel driving cycle. Finally, a series of experiments are executed to prove the effectiveness of the presented control framework. The optimality and adaptability of the formulated EMS are illuminated. The founded DRL and TL-enabled control policy is capable of enhancing energy efficiency and improving system performance.
A comprehensive survey of research towards AI-enabled unmanned aerial systems in pre-, active-, and post-wildfire management
Clemson University（克莱姆森大学）；NSF National Center for Atmospheric Research（美国国家科学基金会国家大气研究中心）；University of San Francisco（旧金山大学）；Northern Arizona University（北亚利桑那大学）；U.S.D.A. Forest Service, Pacific Wildland Fire Science Laboratory（美国农业部林务局太平洋野火科学实验室）；Georgia Institute of Technology（佐治亚理工学院）
University of Liverpool（利物浦大学）；STFC Hartree Centre（STFC哈特里中心）；Defence Science and Technology Laboratory（国防科学与技术实验室）
We present a novel Double Deep Q Network (DDQN) application to a sensor management problem in space situational awareness (SSA). Frequent launches of satellites into Earth orbit pose a significant sensor management challenge, whereby a limited number of sensors are required to detect and track an increasing number of objects. In this paper, we demonstrate the use of reinforcement learning to develop a sensor management policy for SSA. We simulate a controllable Earth-based telescope, which is trained to maximise the number of satellites tracked using an extended Kalman filter. The estimated st…
Blockchain-enabled Federated Learning (BFL) enables mobile devices to collaboratively train neural network models required by a Machine Learning Model Owner (MLMO) while keeping data on the mobile devices. Then, the model updates are stored in the blockchain in a decentralized and reliable manner. However, the issue of BFL is that the mobile devices have energy and CPU constraints that may reduce the system lifetime and training efficiency. The other issue is that the training latency may increase due to the blockchain mining process. To address these issues, the MLMO needs to (i) decide how m…
University of South Florida（南佛罗里达大学）；United States Military Academy, West Point（西点军校）
Cyber vulnerability management is a critical function of a cybersecurity operations center (CSOC) that helps protect organizations against cyber-attacks on their computer and network systems. Adversaries hold an asymmetric advantage over the CSOC, as the number of deficiencies in these systems is increasing at a significantly higher rate compared to the expansion rate of the security teams to mitigate them in a resource-constrained environment. The current approaches are deterministic and one-time decision-making methods, which do not consider future uncertainties when prioritizing and selecti…
With the constant demand for connectivity at an all-time high, Network Service Providers (NSPs) are required to optimize their networks to cope with rising capital and operational expenditures required to meet the growing connectivity demand. A solution to this challenge was presented through Network Function Virtualization (NFV). As network complexity increases and futuristic networks take shape, NSPs are required to incorporate an increasing amount of operational efficiency into their NFV-enabled networks. One such technique is Machine Learning (ML), which has been applied to various entitie…
Multi-Agent DRL for Queue-Aware Task Offloading in Hierarchical MEC-Enabled Air-Ground Networks
Hamad Bin Khalifa University（哈马德·本·哈利法大学）；College of Science and Engineering（科学与工程学院）；Division of Information and Computing Technology（信息与计算技术系）；College of Engineering, Qatar University（卡塔尔大学工程学院）
Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, artificial intelligence (AI) model training, and wireless communications. However, due to the privacy concerns of devices and limited computation or communication resource of UAVs, it is impractical to send raw data of devices to UAV servers for model training. Moreover, due to the dynamic channel condition and heterogeneous computing capacity of devices in UAV-enabled networks, the reliability and efficiency of data sharing require to be further improved. In this paper, we deve…
The sixth generation (6G) network must provide performance superior to previous generations in order to meet the requirements of emerging services and applications, such as multi-gigabit transmission rate, even higher reliability, sub 1 millisecond latency and ubiquitous connection for Internet of Everything. However, with the scarcity of spectrum resources, efficient resource management and sharing is crucial to achieve all these ambitious requirements. One possible technology to enable all of this is blockchain, which has recently gained significance and will be of paramount importance to 6G…
Flexible duplex is proposed to adapt to the channel and traffic asymmetry for future wireless networks. In this paper, we propose two novel algorithms within the flexible duplex framework for joint uplink and downlink resource allocation in multi-cell scenario, named SAFP and RMDI, based on the awareness of interference coupling among wireless links. Numerical results show significant performance gain over the baseline system with fixed uplink/downlink resource configuration, and over the dynamic TDD scheme that independently adapts the configuration to time-varying traffic volume in each cell…
The next generation of wireless communication systems will integrate terrestrial and non-terrestrial networks targeting to cover the undercovered regions, especially connecting the marine activities. Unmanned aerial vehicles (UAVs) based connectivity solutions offer significant advances to support the conventional terrestrial networks. However, the use of UAVs for maritime communication is still an unexplored area of research. Therefore, this paper highlights different aspects of UAV-based maritime communication, including the basic architecture, various channel characteristics, and use cases.…
With the rapid development of smart terminals and infrastructures, as well as diversified applications (e.g., virtual and augmented reality, remote surgery and holographic projection) with colorful requirements, current networks (e.g., 4G and upcoming 5G networks) may not be able to completely meet quickly rising traffic demands. Accordingly, efforts from both industry and academia have already been put to the research on 6G networks. Recently, artificial intelligence (AI) has been utilized as a new paradigm for the design and optimization of 6G networks with a high level of intelligence. Ther…
Thanks to their quick placement and high flexibility, unmanned aerial vehicles (UAVs) can be very useful in the current and future wireless communication systems. With a growing number of smart devices and infrastructure-free communication networks, it is necessary to legitimately monitor these networks to prevent crimes. In this paper, a novel framework is proposed to exploit the flexibility of the UAV for legitimate monitoring via joint trajectory design and energy management. The system includes a suspicious transmission link with a terrestrial transmitter and a terrestrial receiver, and a …
This paper studies a novel approach for successive interference cancellation (SIC) ordering and beamforming in a multiple antennas non-orthogonal multiple access (NOMA) network with multi-carrier multi-user setup. To this end, we formulate a joint beamforming design, subcarrier allocation, user association, and SIC ordering algorithm to maximize the worst-case energy efficiency (EE). The formulated problem is a non-convex mixed integer non-linear programming (MINLP) which is generally difficult to solve. To handle it, we first adopt the linearizion technique as well as relaxing the integer var…
Demand-Side Management (DSM) is a vital tool that can be used to ensure power system reliability and stability. In future smart grids, certain portions of a customers load usage could be under automatic control with a cyber-enabled DSM program which selectively schedules loads as a function of electricity prices to improve power balance and grid stability. In such a case, the security of DSM cyberinfrastructure will be critical as advanced metering infrastructure, and communication systems are susceptible to hacking, cyber-attacks. Such attacks, in the form of data injection, can manipulate cu…
The combined cooling, heating and power (CCHP) system is a typical distributed, electricity-gas integrated energy scheme in a community. First, it generates electricity by use of gas, and then exploits the waste heat to supply community with heat and cooling. In this paper, we consider a smart city consisting of a number of communities (CCHPs) and an agent of power grid (APG), where CCHPs can sell energy to the APG according to its bid. To study all utilities of entities in such a city from energy trading, a noncooperative Stackelberg game between APG and CCHPs is formulated. Here, the APG giv…
The 6G vision is envisaged to enable agile network expansion and rapid deployment of new on-demand microservices (e.g., visibility services for data traffic management, mobile edge computing services) closer to the network's edge IoT devices. However, providing one of the critical features of network visibility services, i.e., data flow prediction in the network, is challenging at the edge devices within a dynamic cloud-native environment as the traffic flow characteristics are random and sporadic. To provide the AI-native services for the 6G vision, we propose a novel edge-native framework to…
From Denoising to Decision Making: A Survey on Diffusion Model-Enabled Deep Reinforcement Learning for Wireless Networks
Phenikaa School of Computing, Phenikaa University（菲尼卡大学计算学院）；School of Computer Science and Electronics Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）；School of Information Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）信息科学与技术学院）；School of Mathematics, Statistics and Mechanics, Beijing University of Technology（北京工业大学数学、统计与力学学院）；Faculty of Artificial Intelligence and Data Science, Phenikaa University（菲尼卡大学人工智能与数据科学学院）；School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University（浙江工商大学信息与电子工程学院（苏塞克斯人工智能研究所））；School of Integrated Circuits, Guangdong University of Technology（广东工业大学集成电路学院）
