arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.04438cs.NI
收藏

DRAMA:一种使用多智能体强化学习与涌现通信的动态数据包路由算法

DRAMA: A Dynamic Packet Routing Algorithm using Multi-Agent Reinforcement Learning with Emergent Communication

Wang Zhang, Chenguang Liu, Yue Pi, Yong Zhang, Hairong Huang, Baoquan Rao, Yulong Ding, Shuanghua Yang, Jie Jiang

AI总结:

本文提出动态路由算法DRAMA,结合多智能体强化学习与涌现通信,引入新Q-network和基于图的通信机制适应动态拓扑,在多种网络场景下实现更高交付率和更低延迟。

摘要:

网络数据的持续扩张给传统路由算法带来了紧迫的挑战。随着需求不断攀升,这些算法正难以应对。在此背景下,强化学习(RL)和多智能体强化学习(MARL)算法作为有前景的解决方案应运而生。然而,问题的紧迫性和重要性显而易见,因为现有的基于RL/MARL的路由方法在运行时缺乏路由器之间的有效通信,使得单个路由器难以适应复杂且动态变化的网络。更重要的是,由于神经网络的不可扩展性,它们缺乏处理动态变化网络拓扑的能力,尤其是路由器的添加。本文提出了一种新颖的动态路由算法DRAMA,在多智能体强化学习中结合了涌现通信。通过涌现通信,路由器可以学习如何有效地进行通信以最大化优化目标。同时,引入了新的Q-network和基于图的涌现通信,以在不重新训练的情况下动态适应变化的网络拓扑,同时确保稳健的性能。实验结果表明,DRAMA在包括动态网络负载和拓扑的多种网络场景中,实现了更高的交付率和更低的延迟,优于传统路由算法和其他基于RL/MARL的算法。此外,消融实验验证了涌现通信在促进数据包路由方面的前景。

英文摘要:

The continuous expansion of network data presents a pressing challenge for conventional routing algorithms. As the demand escalates, these algorithms are struggling to cope. In this context, reinforcement learning (RL) and multi-agent reinforcement learning (MARL) algorithms emerge as promising solutions. However, the urgency and importance of the problem are clear, as existing RL/MARL-based routing approaches lack effective communication in run time among routers, making it challenging for individual routers to adapt to complex and dynamic changing networks. More importantly, they lack the ability to deal with dynamically changing network topology, especially the addition of the router, due to the non-scalability of their neural networks. This paper proposes a novel dynamic routing algorithm, DRAMA, incorporating emergent communication in multi-agent reinforcement learning. Through emergent communication, routers could learn how to communicate effectively to maximize the optimization objectives. Meanwhile, a new Q-network and graph-based emergent communication are introduced to dynamically adapt to the changing network topology without retraining while ensuring robust performance. Experimental results showcase DRAMA's superior performance over the traditional routing algorithm and other RL/MARL-based algorithms, achieving a higher delivery rate and lower latency in diverse network scenarios, including dynamic network load and topology. Moreover, an ablation experiment validates the prospect of emergent communication in facilitating packet routing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑