Constructing earth-fixed cells with low-earth orbit (LEO) satellites in non-terrestrial networks (NTNs) has been the most promising paradigm to enable global coverage. The limited computing capabilities on LEO satellites however render tackling resource optimization within a short duration a critical challenge. Although the sufficient computing capabilities of the ground infrastructures can be utilized to assist the LEO satellite, different time-scale control cycles and coupling decisions between the space- and ground-segments still obstruct the joint optimization design for computing agents at different segments. To address the above challenges, in this paper, a multi-time-scale deep reinforcement learning (DRL) scheme is developed for achieving the radio resource optimization in NTNs, in which the LEO satellite and user equipment (UE) collaborate with each other to perform individual decision-making tasks with different control cycles. Specifically, the UE updates its policy toward improving value functions of both the satellite and UE, while the LEO satellite only performs finite-step rollout for decision-makings based on the reference decision trajectory provided by the UE. Most importantly, rigorous analysis to guarantee the performance convergence of the proposed scheme is provided. Comprehensive simulations are conducted to justify the effectiveness of the proposed scheme in balancing the transmission performance and computational complexity.
Emergency Computing: An Adaptive Collaborative Inference Method Based on Hierarchical Reinforcement Learning
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（北京邮电大学计算机学院（国家示范性软件学院））；School of Electronic Engineering, Beijing University of Posts and Telecommunications（北京邮电大学电子工程学院）
Deep Reinforcement Learning for Interference Suppression in RIS-Aided Space-Air-Ground Integrated Networks
Department of Artificial Intelligence, Indian Institute of Technology Hyderabad, Telangana, India（印度海得拉巴印度理工学院人工智能系）；School of Informatics, Kami, Kochi University of Technology, Kochi, Japan（日本科技大学信息学院）；Graduate School of Information Sciences, Tohoku University, Sendai, Miyagi, Japan（东北大学信息科学研究生院）
Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning
Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）；The Chinese University of Hong Kong（香港中文大学）；Hainan University（海南大学）；Southeast University（东南大学）；Shanghai Jiao Tong University（上海交通大学）
