arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.04041cs.NIcs.DCcs.ITcs.LGcs.MAmath.IT
收藏

分层深度双Q路由

Hierarchical Deep Double Q-Routing

Penn State University(宾夕法尼亚州立大学);Nokia Bell Labs(诺基亚贝尔实验室)

Ramy E. Ali, Bilgehan Erman, Ejder Baştuğ, Bruce Cilli

摘要:

本文探索了一种深度强化学习方法,应用于动态和自治通信网络引发的高维约束的数据包路由问题。我们的方法的动机是这样一个事实,即集中式路径计算方法往往是不可扩展的,而本地代理节点的分布式方法并不完全知道端到端的性能。相反,我们将路径计算分层分布在网络中的指定节点上,同时考虑到端到端的性能。具体来说,我们开发了一个层次集群导向的自适应每流路径计算机制,利用深度双Q网络(DDQN)算法,其中端到端的路径计算的源节点的协助下,集群(组)领导人在不同层次。在我们的方法中,延迟的复合奖励被设计成通过从源节点到组领导者的反馈信号来捕获端到端的性能,并通过组领导者的本地资源评估来捕获本地网络性能。该方法可在大型网络中扩展,适应动态需求,有效利用网络资源,适用于分段路由。

英文摘要:

This paper explores a deep reinforcement learning approach applied to the packet routing problem with high-dimensional constraints instigated by dynamic and autonomous communication networks. Our approach is motivated by the fact that centralized path calculation approaches are often not scalable, whereas the distributed approaches with locally acting nodes are not fully aware of the end-to-end performance. We instead hierarchically distribute the path calculation over designated nodes in the network while taking into account the end-to-end performance. Specifically, we develop a hierarchical cluster-oriented adaptive per-flow path calculation mechanism by leveraging the Deep Double Q-network (DDQN) algorithm, where the end-to-end paths are calculated by the source nodes with the assistance of cluster (group) leaders at different hierarchical levels. In our approach, a deferred composite reward is designed to capture the end-to-end performance through a feedback signal from the source nodes to the group leaders and captures the local network performance through the local resource assessments by the group leaders. This approach scales in large networks, adapts to the dynamic demand, utilizes the network resources efficiently and can be applied to segment routing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑