arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.06185cs.ITmath.IT
收藏

部分可观测状态下能量收集两跳通信的多智能体强化学习

Multi-Agent Reinforcement Learning for Energy Harvesting Two-Hop Communications with a Partially Observable State

Andrea Ortiz, Hussein Al-Shatri, Tobias Weber, Anja Klein

摘要:

我们考虑一个能量收集(EH)发射机通过EH继电器与接收机通信。收集的能量用于数据传输,包括电路能量消耗。如在实际场景中,由所收集的能量、电池水平、数据缓冲器水平和信道增益组成的系统状态仅部分地可由EH节点观察到。此外,EH节点仅具有关于它们自己的发送信道的信道增益的过时知识。我们的目标是找到分布式传输策略,旨在最大限度地提高吞吐量。在每个EH节点中实现基于卡尔曼滤波器的信道预测器,以估计其自身信道的当前信道增益。此外,为了克服系统状态的部分可观测性,EH节点彼此合作以在信令阶段期间获得关于它们的参数的信息。我们将问题建模为马尔可夫博弈,并提出了一种多智能体强化学习算法来寻找传输策略。我们展示了可实现的吞吐量和所需的信令之间的权衡,并为所提出的算法提供收敛保证。结果表明,即使当信令开销被考虑在内,该算法优于其他方法,不考虑合作。

英文摘要:

We consider an energy harvesting (EH) transmitter communicating with a receiver through an EH relay. The harvested energy is used for data transmission, including the circuit energy consumption. As in practical scenarios, the system state, comprised by the harvested energy, battery levels, data buffer levels, and channel gains, is only partially observable by the EH nodes. Moreover, the EH nodes have only outdated knowledge regarding the channel gains for their own transmit channels. Our goal is to find distributed transmission policies aiming at maximizing the throughput. A channel predictor based on a Kalman filter is implemented in each EH node to estimate the current channel gain for its own channel. Furthermore, to overcome the partial observability of the system state, the EH nodes cooperate with each other to obtain information about their parameters during a signaling phase. We model the problem as a Markov game and propose a multi-agent reinforcement learning algorithm to find the transmission policies. We show the trade-off between the achievable throughput and the signaling required, and provide convergence guarantees for the proposed algorithm. Results show that even when the signaling overhead is taken into account, the proposed algorithm outperforms other approaches that do not consider cooperation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑