arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.11047cs.LGcs.NIeess.SP
收藏

面向卫星网络的开源多智能体深度强化学习路由模拟器

An open source Multi-Agent Deep Reinforcement Learning Routing Simulator for satellite networks

Telecommunications Research Institute, University of Malaga(马拉加大学电信研究院);Aalborg University(奥尔堡大学)

Federico Lozano-Cuadra, Mathias D. Thorsager, Israel Leyva-Mayorga, Beatriz Soret

AI总结:

本文针对低地球轨道卫星星座动态系统不确定性问题,提出开源Python模拟器,支持传统路由与Q-Routing、MA-DRL等学习方案,通过事件驱动模拟提升端到端延迟性能,代码文档开源。

摘要:

本文介绍了一款针对低地球轨道卫星星座(LSatCs)中数据包路由的开源模拟器,该模拟器考虑了动态系统不确定性。此模拟器基于Python实现,支持传统的Dijkstra算法路由以及更先进的学习解决方案,特别是我们先前工作中的Q-Routing和多智能体深度强化学习(MA-DRL)。它采用基于SimPy模块的事件驱动方法,精确模拟数据包的生成、路由和排队过程,并提供队列和延迟的实时跟踪功能。该模拟器具有高度可配置性,允许调整路由策略、流量、地面与空间层拓扑、通信参数以及学习超参数。其关键特性包括系统运动可视化能力和数据包路径跟踪功能。结果表明,与传统方法相比,基于强化学习(RL)的路由策略在端到端(E2E)延迟方面有显著改善。源代码、文档以及包含后处理结果和分析的Jupyter笔记本可在GitHub上获取。

英文摘要:

This paper introduces an open source simulator for packet routing in Low Earth Orbit Satellite Constellations (LSatCs) considering the dynamic system uncertainties. The simulator, implemented in Python, supports traditional Dijkstra's based routing as well as more advanced learning solutions, specifically Q-Routing and Multi-Agent Deep Reinforcement Learning (MA-DRL) from our previous work. It uses an event-based approach with the SimPy module to accurately simulate packet creation, routing and queuing, providing real-time tracking of queues and latency. The simulator is highly configurable, allowing adjustments in routing policies, traffic, ground and space layer topologies, communication parameters, and learning hyperparameters. Key features include the ability to visualize system motion and track packet paths. Results highlight significant improvements in end-to-end (E2E) latency using Reinforcement Learning (RL)-based routing policies compared to traditional methods. The source code, the documentation and a Jupyter notebook with post-processing results and analysis are available on GitHub.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑