Reinforcement Learning (RL) has gained significant momentum in the development of network protocols. However, RL-based protocols are still in their infancy, and substantial research is required to build deployable solutions. Developing a protocol based on RL is a complex and challenging process that involves several model design decisions and requires significant training and evaluation in real and simulated network topologies. Network simulators offer an efficient training environment for RL-based protocols, because they are deterministic and can run in parallel. In this paper, we introduce \textit{RayNet}, a scalable and adaptable simulation platform for the development of RL-based network protocols. RayNet integrates OMNeT++, a fully programmable network simulator, with Ray/RLlib, a scalable training platform for distributed RL. RayNet facilitates the methodical development of RL-based network protocols so that researchers can focus on the problem at hand and not on implementation details of the learning aspect of their research. We developed a simple RL-based congestion control approach as a proof of concept showcasing that RayNet can be a valuable platform for RL-based research in computer networks, enabling scalable training and evaluation. We compared RayNet with \textit{ns3-gym}, a platform with similar objectives to RayNet, and showed that RayNet performs better in terms of how fast agents can collect experience in RL environments.
Developing a Trusted Human-AI Network for Humanitarian Benefit
Trusted Autonomous Systems（可信自主系统）；University of Queensland（昆士兰大学）；RMIT University（皇家墨尔本理工大学）；Whiteflag Foundation（白旗基金会）；Center for Naval Analysis（海军分析中心）
MEEP: Multihop Energy Efficient Protocol For Heterogeneous Wireless Sensor Network
在无线传感器网络中，节点能量的节约是提高网络寿命的关键。最近的技术进步有助于开发各种各样的传感器节点。异构网络充分利用了不同类型的传感器节点，提高了网络的能量效率和寿命。传感器网络通常是随机密集部署在一个传感区域内，因此短距离多跳通信减少了传感器网络中的长距离传输。本文提出了一种异构传感器网络多跳节能协议MEEP（multihop energy efficient protocol for heterogeneous sensor network）。该协议结合了分簇和多跳通信的思想。通过使用一些高能量的节点在网络中创建异构。低能量节点使用基于剩余能量的方案成为簇头。高能量节点在不履行簇头职责时充当低能量簇头的中继节点，以进一步节省能量。协议还建议在簇形成过程中节点处于睡眠状态，以节省能量，增加传感器网络的寿命。仿真结果表明，该方案在能量效率和网络寿命方面优于SEP等两级异构传感器网络协议。
