The Open Radio Access Network (Open RAN) paradigm, and its reference architecture proposed by the O-RAN Alliance, is paving the way toward open, interoperable, observable and truly intelligent cellular networks. Crucial to this evolution is Machine Learning (ML), which will play a pivotal role by providing the necessary tools to realize the vision of self-organizing O-RAN systems. However, to be actionable, ML algorithms need to demonstrate high reliability, effectiveness in delivering high performance, and the ability to adapt to varying network conditions, traffic demands and performance requirements. To address these challenges, in this paper we propose a novel Deep Reinforcement Learning (DRL) agent design for O-RAN applications that can learn control policies under varying Service Level Agreement (SLAs) with heterogeneous minimum performance requirements. We focus on the case of RAN slicing and SLAs specifying maximum tolerable end-to-end latency levels. We use the OpenRAN Gym open-source environment to train a DRL agent that can adapt to varying SLAs and compare it against the state-of-the-art. We show that our agent maintains a low SLA violation rate that is 8.3x and 14.4x lower than approaches based on Deep Q- Learning (DQN) and Q-Learning while consuming respectively 0.3x and 0.6x fewer resources without the need for re-training.
5G and beyond is expected to enable various emerging use cases with diverse performance requirements from vertical industries. To serve these use cases cost-effectively, network slicing plays a key role in dynamically creating virtual end-to-end networks according to specific resource demands. A network slice may have hundreds of configurable parameters over multiple technical domains that define the performance of the network slice, which makes it impossible to use traditional model-based solutions to orchestrate resources for network slices. In this article, we discuss how to design and depl…
The complexity of emerging sixth-generation (6G) wireless networks has sparked an upsurge in adopting artificial intelligence (AI) to underpin the challenges in network management and resource allocation under strict service level agreements (SLAs). It inaugurates the era of massive network slicing as a distributive technology where tenancy would be extended to the final consumer through pervading the digitalization of vertical immersive use-cases. Despite the promising performance of deep reinforcement learning (DRL) in network slicing, lack of transparency, interpretability, and opaque model…
Zhejiang University（浙江大学）；China Mobile Research Institute（中国移动研究院）；Beihang University（北京航空航天大学）；VTT Technical Research Centre of Finland（芬兰VTT技术研究中心）
Network slicing is born as an emerging business to operators, by allowing them to sell the customized slices to various tenants at different prices. In order to provide better-performing and cost-efficient services, network slicing involves challenging technical issues and urgently looks forward to intelligent innovations to make the resource management consistent with users' activities per slice. In that regard, deep reinforcement learning (DRL), which focuses on how to interact with the environment by trying alternative actions and reinforcing the tendency actions producing more rewarding co…
LEO-Aware DRL Meta-Scheduler for 5G Non-Terrestrial Network Slicing
Professional School of Computing Science, National University of San Agustin（圣奥古斯丁国立大学计算机科学专业学院）；Institute of Computing, University of Campinas（坎皮纳斯大学计算研究所）
Artificial intelligence (AI)-driven zero-touch network slicing (NS) is a new paradigm enabling the automation of resource management and orchestration (MANO) in multi-tenant beyond 5G (B5G) networks. In this paper, we tackle the problem of cloud-RAN (C-RAN) joint slice admission control and resource allocation by first formulating it as a Markov decision process (MDP). We then invoke an advanced continuous deep reinforcement learning (DRL) method called twin delayed deep deterministic policy gradient (TD3) to solve it. In this intent, we introduce a multi-objective approach to make the central…
Network slicing is a critical driver for guaranteeing the diverse service level agreements (SLA) in 5G and future networks. Recently, deep reinforcement learning (DRL) has been widely utilized for resource allocation in network slicing. However, existing related works do not consider the performance loss associated with the initial exploration phase of DRL. This paper proposes a new performance-guaranteed slicing strategy with a soft and hard hybrid slicing setting. Mainly, a common slice setting is applied to guarantee slices' SLA when training the neural network. Moreover, the resource of th…
Network slicing enables multiple virtual networks to be instantiated and customized to meet heterogeneous use case requirements over 5G and beyond network deployments. However, most of the solutions available today face scalability issues when considering many slices, due to centralized controllers requiring a holistic view of the resource availability and consumption over different networking domains. In order to tackle this challenge, we design a hierarchical architecture to manage network slices resources in a federated manner. Driven by the rapid evolution of deep reinforcement learning (D…
LLM-Empowered Agentic AI for QoE-Aware Network Slicing Management in Industrial IoT
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（北京邮电大学网络与交换技术国家重点实验室）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Department of Electrical and Electronic Engineering, University of Hong Kong（香港大学电子电气工程学系）；School of Electrical and Computer Engineering, University of Sydney（悉尼大学电气与计算机工程学院）
University of Nebraska-Lincoln（内布拉斯加大学林肯分校）；Nokia Bell Labs（诺基亚贝尔实验室）；New Jersey Institute of Technology（新泽西理工学院）
Network slicing allows mobile network operators to virtualize infrastructures and provide customized slices for supporting various use cases with heterogeneous requirements. Online deep reinforcement learning (DRL) has shown promising potential in solving network problems and eliminating the simulation-to-reality discrepancy. Optimizing cross-domain resources with online DRL is, however, challenging, as the random exploration of DRL violates the service level agreement (SLA) of slices and resource constraints of infrastructures. In this paper, we propose OnSlicing, an online end-to-end network…
5G and edge computing will serve various emerging use cases that have diverse requirements of multiple resources, e.g., radio, transportation, and computing. Network slicing is a promising technology for creating virtual networks that can be customized according to the requirements of different use cases. Provisioning network slices requires end-to-end resource orchestration which is challenging. In this paper, we design a decentralized resource orchestration system named EdgeSlice for dynamic end-to-end network slicing. EdgeSlice introduces a new decentralized deep reinforcement learning (D-D…
Network slicing is an effective 5G concept for improved resource utilization and service scalability tailored to users (UEs) requirements. According to the standardization, 5G system should support UEs through specification of its heterogenous requirements such as, high data rates, traffic density, latency, reliability, UE density, system efficiency and service availability. These requirements are specified as Service Level Agreements (SLAs) between UEs and network operators resulting in increased interest to develop novel challenging mechanisms for improved interference-free resource efficien…
This paper studies the multi-agent resource allocation problem in vehicular networks using non-orthogonal multiple access (NOMA) and network slicing. To ensure heterogeneous service requirements for different vehicles, we propose a network slicing architecture. We focus on a non-cellular network scenario where vehicles communicate by the broadcast approach via the direct device-to-device interface. In such a vehicular network, resource allocation among vehicles is very difficult, mainly due to (i) the rapid variation of wireless channels among highly mobile vehicles and (ii) the lack of a cent…
University of South Florida（南佛罗里达大学）；Electrical Engineering Department, University of South Florida（南佛罗里达大学电气工程系）；University of South Florida, Tampa, FL 33620, USA（南佛罗里达大学，美国佛罗里达州坦帕市，邮编33620）
Intelligent vehicular systems and smart city applications are the fastest growing Internet of things (IoT) implementations at a compound annual growth rate of 30%. In view of the recent advances in IoT devices and the emerging new breed of IoT applications driven by artificial intelligence (AI), fog radio access network (F-RAN) has been recently introduced for the fifth generation (5G) wireless communications to overcome the latency limitations of cloud-RAN (C-RAN). We consider the network slicing problem of allocating the limited resources at the network edge (fog nodes) to vehicular and smar…
The interconnection of vehicles in the future fifth generation (5G) wireless ecosystem forms the so-called Internet of vehicles (IoV). IoV offers new kinds of applications requiring delay-sensitive, compute-intensive and bandwidth-hungry services. Mobile edge computing (MEC) and network slicing (NS) are two of the key enabler technologies in 5G networks that can be used to optimize the allocation of the network resources and guarantee the diverse requirements of IoV applications. As traditional model-based optimization techniques generally end up with NP-hard and strongly non-convex and non-li…
To harness the full potential of beyond 5G (B5G) communication systems, zero-touch network slicing (NS) is viewed as a promising fully-automated management and orchestration (MANO) system. This paper proposes a novel knowledge plane (KP)-based MANO framework that accommodates and exploits recent NS technologies and is termed KB5G. Specifically, we deliberate on algorithmic innovation and artificial intelligence (AI) in KB5G. We invoke a continuous model-free deep reinforcement learning (DRL) method to minimize energy consumption and virtual network function (VNF) instantiation cost. We present…
Queen’s University（女王大学）；Cairo University（开罗大学）；University of Calgary（卡尔加里大学）
Deep reinforcement learning (DRL) algorithms have recently gained wide attention in the wireless networks domain. They are considered promising approaches for solving dynamic radio resource management (RRM) problems in next-generation networks. Given their capabilities to build an approximate and continuously updated model of the wireless network environments, DRL algorithms can deal with the multifaceted complexity of such environments. Nevertheless, several challenges hinder the practical adoption of DRL in commercial networks. In this article, we first discuss two key practical challenges t…
Artificial intelligence (AI)-driven zero-touch network slicing is envisaged as a promising cutting-edge technology to harness the full potential of heterogeneous 5G and beyond 5G (B5G) communication systems and enable the automation of demand-aware resource management and orchestration (MANO). In this paper, we tackle the issue of B5G radio access network (RAN) joint slice admission control and resource allocation according to proposed slice-enabling cell-free massive multiple-input multiple-output (mMIMO) setup by invoking a continuous deep reinforcement learning (DRL) method. We present a no…
Artificial intelligence (AI)-driven zero-touch massive network slicing is envisioned to be a disruptive technology in beyond 5G (B5G)/6G, where tenancy would be extended to the final consumer in the form of advanced digital use-cases. In this paper, we propose a novel model-free deep reinforcement learning (DRL) framework, called collaborative statistical Actor-Critic (CS-AC) that enables a scalable and farsighted slice performance management in a 6G-like RAN scenario that is built upon mobile edge computing (MEC) and massive multiple-input multiple-output (mMIMO). In this intent, the proposed…
Technische Universität Kaiserslautern（卡尔斯鲁厄理工大学）；NEC Laboratories Europe GmbH.（欧洲电气股份有限公司实验室）
Network Slicing is expected to become a game changer in the upcoming 5G networks and beyond, enlarging the telecom business ecosystem through still-unexplored vertical industry profits. This implies that heterogeneous service level agreements (SLAs) must be guaranteed per slice given the multitude of predefined requirements. In this paper, we pioneer a novel radio slicing orchestration solution that simultaneously provides-latency and throughput guarantees in a multi-tenancy environment. Leveraging on a solid mathematical framework, we exploit the exploration-vs-exploitation paradigm by means …
Network slicing enables the operator to configure virtual network instances for diverse services with specific requirements. To achieve the slice-aware radio resource scheduling, dynamic slicing resource partitioning is needed to orchestrate multi-cell slice resources and mitigate inter-cell interference. It is, however, challenging to derive the analytical solutions due to the complex inter-cell interdependencies, interslice resource constraints, and service-specific requirements. In this paper, we propose a multi-agent deep reinforcement learning (DRL) approach that improves the max-min slic…
Network slicing is a promising technology that allows mobile network operators to efficiently serve various emerging use cases in 5G. It is challenging to optimize the utilization of network infrastructures while guaranteeing the performance of network slices according to service level agreements (SLAs). To solve this problem, we propose SafeSlicing that introduces a new constraint-aware deep reinforcement learning (CaDRL) algorithm to learn the optimal resource orchestration policy within two steps, i.e., offline training in a simulated environment and online learning with the real network sy…
We consider the tradeoff between resource efficiency and performance isolation that emerges when multiplexing the resource demands of Network Slices (NSs). On the one hand, multiplexing allows the use of idle resources, which increases resource efficiency. On the other hand, the performance of each NS becomes susceptible to traffic surges in other NSs, which degrades performance isolation. The analysis of this tradeoff enables network operators to determine the effect of performance isolation on the operating cost of each NS. To study the tradeoff, we solve an optimization problem where we fin…
Telecommunications Technological Center of Catalonia (CTTC)（加泰罗尼亚电信技术中心）；Technical University of Catalonia (UPC)（加泰罗尼亚理工大学）；i2CAT Foundation（i2CAT基金会）
The quantum machine learning (QML) paradigms and their synergies with network slicing can be envisioned to be a disruptive technology on the cusp of entering to era of sixth-generation (6G), where the mobile communication systems are underpinned in the form of advanced tenancy-based digital use-cases to meet different service requirements. To overcome the challenges of massive slices such as handling the increased dynamism, heterogeneity, amount of data, extended training time, and variety of security levels for slice instances, the power of quantum computing pursuing a distributed computation…
Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）；VTT Technical Research Centre of Finland（芬兰VTT技术研究中心）
Network slicing is a key technology in 5G communications system. Its purpose is to dynamically and efficiently allocate resources for diversified services with distinct requirements over a common underlying physical infrastructure. Therein, demand-aware resource allocation is of significant importance to network slicing. In this paper, we consider a scenario that contains several slices in a radio access network with base stations that share the same physical resources (e.g., bandwidth or slots). We leverage deep reinforcement learning (DRL) to solve this problem by considering the varying ser…
In spite of the new opportunities brought about by the Open RAN, advances in ML-based network automation have been slow, mainly because of the unavailability of large-scale datasets and experimental testing infrastructure. This slows down the development and widespread adoption of Deep Reinforcement Learning (DRL) agents on real networks, delaying progress in intelligent and autonomous RAN control. In this paper, we address these challenges by proposing practical solutions and software pipelines for the design, training, testing, and experimental evaluation of DRL-based closed-loop control in …
