arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.10883cs.NIcs.LG
收藏

ENERO:利用深度强化学习进行高效实时的WAN路由优化

ENERO: Efficient Real-Time WAN Routing Optimization with Deep Reinforcement Learning

Barcelona Neural Networking Center, Universitat Politècnica de Catalunya(加泰罗尼亚理工大学巴塞罗那神经网络中心);Huawei Technologies Co.,Ltd.(华为技术有限公司)

Paul Almasan, Shihan Xiao, Xiangle Cheng, Xiang Shi, Pere Barlet-Ros, Albert Cabellos-Aparicio

摘要:

广域网(WAN)是当今社会的关键基础设施。在过去的几年中,WAN已经看到网络的流量和网络应用的相当大的增长,对现有的网络技术(例如,低等待时间和高吞吐量)。因此,互联网服务提供商(ISP)面临着确保客户的服务质量和履行服务水平协议的压力。网络运营商利用流量工程(TE)技术来有效地管理网络资源。然而,WAN的流量可能会随着时间的推移而急剧变化,并且连接性可能会因外部因素(例如,链路故障)。因此,TE解决方案必须能够实时的【形】适应动态场景。在本文中,我们提出了Enero,一个有效的实时的【形】TE解决方案的基础上,两个阶段的优化过程。在第一个中,Enero利用深度强化学习(DRL)通过生成长期TE策略来优化路由配置。为了能够在动态网络场景(例如,当发生链路故障时),我们将图神经网络集成到DRL代理中。在第二阶段,Enero使用局部搜索算法来改进DRL的解决方案,而不会增加优化过程的计算开销。实验结果表明,Enero能够在真实世界的动态网络拓扑结构中运行,平均4.5秒的拓扑结构多达100个边缘。

英文摘要:

Wide Area Networks (WAN) are a key infrastructure in today's society. During the last years, WANs have seen a considerable increase in network's traffic and network applications, imposing new requirements on existing network technologies (e.g., low latency and high throughput). Consequently, Internet Service Providers (ISP) are under pressure to ensure the customer's Quality of Service and fulfill Service Level Agreements. Network operators leverage Traffic Engineering (TE) techniques to efficiently manage network's resources. However, WAN's traffic can drastically change during time and the connectivity can be affected due to external factors (e.g., link failures). Therefore, TE solutions must be able to adapt to dynamic scenarios in real-time. In this paper we propose Enero, an efficient real-time TE solution based on a two-stage optimization process. In the first one, Enero leverages Deep Reinforcement Learning (DRL) to optimize the routing configuration by generating a long-term TE strategy. To enable efficient operation over dynamic network scenarios (e.g., when link failures occur), we integrated a Graph Neural Network into the DRL agent. In the second stage, Enero uses a Local Search algorithm to improve DRL's solution without adding computational overhead to the optimization process. The experimental results indicate that Enero is able to operate in real-world dynamic network topologies in 4.5 seconds on average for topologies up to 100 edges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑