arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1906.08792cs.LGcs.ITmath.ITstat.ML
收藏

当多个代理学习调度时:一个分布式无线资源管理框架

When Multiple Agents Learn to Schedule: A Distributed Radio Resource Management Framework

Intel Corporation(英特尔公司)

Navid Naderializadeh, Jaroslaw Sydir, Meryem Simsek, Hosein Nikopour, Shilpa Talwar

摘要:

在无线网络中,并发传输之间的干扰是限制系统性能的关键因素。缓解该问题的一种方式是管理无线电资源,以便最大化平均或最坏情况下的性能。然而,这两个指标的联合考虑往往被忽视,因为它们在本质上是竞争的。在这篇文章中,提出了一种使用多代理深度强化学习(RL)的无线电资源管理机制,该机制在最大化平均和$5^{th}$百分位用户吞吐量之间取得了正确的权衡。网络中的每个发射机都配备有深度RL代理,从网络接收部分观测(例如,信道质量、干扰电平等)以及对于给定的无线电资源,在每个调度间隔决定是活动的还是不活动的,该过程被称为链路调度。根据所有代理的行为,网络向代理发出奖励,表明他们的联合决策有多好。建议的框架使代理以分布式的方式作出决定,和奖励的设计方式,代理努力保证最低的性能,导致公平的资源分配之间的所有用户在整个网络。仿真结果表明,我们的方法相比,分散的基线的平均和$5^{th}$百分位用户吞吐量的优越性,同时实现性能接近的集中式穷举搜索方法。此外,所提出的框架是强大的训练和测试场景之间的不匹配。特别是,它示出了在具有低发射器密度的网络上训练的代理保持其性能,并且当部署在具有较高发射器密度的网络中时优于基线。

英文摘要:

Interference among concurrent transmissions in a wireless network is a key factor limiting the system performance. One way to alleviate this problem is to manage the radio resources in order to maximize either the average or the worst-case performance. However, joint consideration of both metrics is often neglected as they are competing in nature. In this article, a mechanism for radio resource management using multi-agent deep reinforcement learning (RL) is proposed, which strikes the right trade-off between maximizing the average and the $5^{th}$ percentile user throughput. Each transmitter in the network is equipped with a deep RL agent, receiving partial observations from the network (e.g., channel quality, interference level, etc.) and deciding whether to be active or inactive at each scheduling interval for given radio resources, a process referred to as link scheduling. Based on the actions of all agents, the network emits a reward to the agents, indicating how good their joint decisions were. The proposed framework enables the agents to make decisions in a distributed manner, and the reward is designed in such a way that the agents strive to guarantee a minimum performance, leading to a fair resource allocation among all users across the network. Simulation results demonstrate the superiority of our approach compared to decentralized baselines in terms of average and $5^{th}$ percentile user throughput, while achieving performance close to that of a centralized exhaustive search approach. Moreover, the proposed framework is robust to mismatches between training and testing scenarios. In particular, it is shown that an agent trained on a network with low transmitter density maintains its performance and outperforms the baselines when deployed in a network with a higher transmitter density.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑