arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2111.01616cs.NIcs.LG
收藏

OnSlicing:基于强化学习的在线端到端网络切片

OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning

University of Nebraska-Lincoln(内布拉斯加大学林肯分校);Nokia Bell Labs(诺基亚贝尔实验室);New Jersey Institute of Technology(新泽西理工学院)

Qiang Liu, Nakjung Choi, Tao Han

摘要:

网络切片允许移动网络运营商虚拟化基础设施并提供定制切片,以支持具有异构需求的各种用例。在线深度强化学习(DRL)在解决网络问题和消除模拟与现实的差异方面显示出很大的潜力。然而,使用在线DRL优化跨域资源是具有挑战性的,因为DRL的随机探索违反了切片的服务水平协议(SLA)和基础设施的资源约束。在本文中,我们提出了OnSlicing,一个在线的端到端的网络切片系统,以实现最小的资源使用,同时满足切片的SLA。OnSlicing允许对每个切片进行个性化学习,并通过使用一种新的约束感知策略更新方法和主动基线切换机制来维护其SLA。OnSlicing通过使用切片中的动作修改和基础设施中的参数协调的独特设计来遵守基础设施的资源约束。OnSlicing通过离线模仿基于规则的解决方案,进一步缓解了早期学习阶段在线学习的不良性能。此外,我们设计了四个新的域管理器,使动态资源配置在无线接入,传输,核心和边缘网络,分别在亚秒的时间尺度。我们在基于OpenAirInterface设计的端到端切片测试平台上实现了OnSlicing,该测试平台具有4G LTE和5G NR,OpenDayLight SDN平台和OpenAir-CN核心网络。实验结果表明,OnSlicing实现了61.3%的使用减少相比,基于规则的解决方案,并保持在整个在线学习阶段几乎为零违规(0.06%)。随着在线学习的融合,与最先进的在线DRL解决方案相比,OnSlicing减少了12.5%的使用量,而没有任何违规行为。

英文摘要:

Network slicing allows mobile network operators to virtualize infrastructures and provide customized slices for supporting various use cases with heterogeneous requirements. Online deep reinforcement learning (DRL) has shown promising potential in solving network problems and eliminating the simulation-to-reality discrepancy. Optimizing cross-domain resources with online DRL is, however, challenging, as the random exploration of DRL violates the service level agreement (SLA) of slices and resource constraints of infrastructures. In this paper, we propose OnSlicing, an online end-to-end network slicing system, to achieve minimal resource usage while satisfying slices' SLA. OnSlicing allows individualized learning for each slice and maintains its SLA by using a novel constraint-aware policy update method and proactive baseline switching mechanism. OnSlicing complies with resource constraints of infrastructures by using a unique design of action modification in slices and parameter coordination in infrastructures. OnSlicing further mitigates the poor performance of online learning during the early learning stage by offline imitating a rule-based solution. Besides, we design four new domain managers to enable dynamic resource configuration in radio access, transport, core, and edge networks, respectively, at a timescale of subseconds. We implement OnSlicing on an end-to-end slicing testbed designed based on OpenAirInterface with both 4G LTE and 5G NR, OpenDayLight SDN platform, and OpenAir-CN core network. The experimental results show that OnSlicing achieves 61.3% usage reduction as compared to the rule-based solution and maintains nearly zero violation (0.06%) throughout the online learning phase. As online learning is converged, OnSlicing reduces 12.5% usage without any violations as compared to the state-of-the-art online DRL solution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑