arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1905.03494cs.NIcs.AI
收藏

基于全分布式多智能体深度强化学习的分组路由

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

Fudan University(复旦大学);East China University of Science and Technology(华东理工大学);Hubei Normal University(湖北师范大学)

Xinyu You, Xuanjie Li, Yuedong Xu, Hui Feng, Jin Zhao, Huaicheng Yan

摘要:

分组路由是计算机网络中的基本问题之一,其中路由器确定队列中每个分组的下一跳以使其尽可能快地到达目的地。强化学习(Reinforcement learning,RL)被引入到利用随机分组到达和服务的本地信息来设计自治分组路由策略中。然而,RL的维数灾难禁止更全面的动态网络状态的表示,从而限制了其潜在的好处。在本文中,我们提出了一种新的基于\emph{multi-agent}深度强化学习(DRL)的数据包路由框架,其中每个路由器都拥有一个\emph{independent} LSTM递归神经网络,用于在\emph{fully distributed}环境中进行训练和决策。LSTM递归神经网络从关于积压数据包和过去操作的丰富信息中提取路由特征,并有效地逼近Q学习的价值函数。我们还允许每个路由定期与直接邻居进行通信,以便更广泛地了解网络状态。实验结果表明,我们的多代理DRL政策可以罢工之间的微妙平衡,感知和最短的路由,并显着减少数据包的交付时间在一般的网络拓扑结构相比,它的同行。

英文摘要:

Packet routing is one of the fundamental problems in computer networks in which a router determines the next-hop of each packet in the queue to get it as quickly as possible to its destination. Reinforcement learning (RL) has been introduced to design autonomous packet routing policies with local information of stochastic packet arrival and service. However, the curse of dimensionality of RL prohibits the more comprehensive representation of dynamic network states, thus limiting its potential benefit. In this paper, we propose a novel packet routing framework based on \emph{multi-agent} deep reinforcement learning (DRL) in which each router possess an \emph{independent} LSTM recurrent neural network for training and decision making in a \emph{fully distributed} environment. The LSTM recurrent neural network extracts routing features from rich information regarding backlogged packets and past actions, and effectively approximates the value function of Q-learning. We further allow each route to communicate periodically with direct neighbors so that a broader view of network state can be incorporated. Experimental results manifest that our multi-agent DRL policy can strike the delicate balance between congestion-aware and shortest routes, and significantly reduce the packet delivery time in general network topologies compared with its counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑