arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.04986cs.ITcs.LGmath.IT
收藏

多智能体深度强化学习(MADRL)满足多用户MIMO系统

Multi-agent deep reinforcement learning (MADRL) meets multi-user MIMO systems

Ericsson Research, Ericsson AB(爱立信研究,爱立信公司)

Heunchul Lee, Jaeseong Jeong

摘要:

多智能体深度强化学习(MADRL)是解决无线环境中涉及多个决策者(或行动者)和高维连续动作空间的挑战性问题的一种有前途的方法。在本文中,我们提出了一种基于MADRL的方法,可以联合优化预编码器,以实现多输入单输出(MISO)干扰信道(IFC)的可实现速率区域的外边界,称为帕累托边界。为了解决两个主要的挑战,即,多个行动者(或代理)的部分可观察性和多维连续的行动空间在MISO IFC设置,我们采用了多代理深度确定性策略梯度(MA-DDPG)框架中,分散的部分可观察性的行动者可以学习一个多维连续的策略,在一个集中的方式与援助的共享评论家与全球信息。同时,我们还将解决在无线电通信中广泛使用的信号的传统复基带表示的相位模糊问题。为了减轻相位模糊对训练性能的影响,我们提出了一种训练方法,称为相位模糊消除(PAE),导致更快的学习和更好的性能的MA-DDPG在无线通信系统中。仿真结果表明,MA-DDPG是能够学习一个接近最佳的预编码策略在MISO IFC环境。据我们所知,这是第一个工作,证明了MA-DDPG框架可以联合优化预编码器,以实现在多小区多用户多天线系统中的可实现速率区域的帕累托边界。

英文摘要:

A multi-agent deep reinforcement learning (MADRL) is a promising approach to challenging problems in wireless environments involving multiple decision-makers (or actors) with high-dimensional continuous action space. In this paper, we present a MADRL-based approach that can jointly optimize precoders to achieve the outer-boundary, called pareto-boundary, of the achievable rate region for a multiple-input single-output (MISO) interference channel (IFC). In order to address two main challenges, namely, multiple actors (or agents) with partial observability and multi-dimensional continuous action space in MISO IFC setup, we adopt a multi-agent deep deterministic policy gradient (MA-DDPG) framework in which decentralized actors with partial observability can learn a multi-dimensional continuous policy in a centralized manner with the aid of shared critic with global information. Meanwhile, we will also address a phase ambiguity issue with the conventional complex baseband representation of signals widely used in radio communications. In order to mitigate the impact of phase ambiguity on training performance, we propose a training method, called phase ambiguity elimination (PAE), that leads to faster learning and better performance of MA-DDPG in wireless communication systems. The simulation results exhibit that MA-DDPG is capable of learning a near-optimal precoding strategy in a MISO IFC environment. To the best of our knowledge, this is the first work to demonstrate that the MA-DDPG framework can jointly optimize precoders to achieve the pareto-boundary of achievable rate region in a multi-cell multi-user multi-antenna system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑