信号衰减使网络上的可扩展去中心化多智能体强化学习成为可能
Signal attenuation enables scalable decentralized multi-agent reinforcement learning over networks
AI总结：
本文利用无线通信中的信号衰减特性，通过提出受约束的多智能体马尔可夫决策过程公式和去中心化鞍点策略梯度算法，实现了基于局部邻域可观测性的可扩展去中心化多智能体强化学习。
摘要：
多智能体强化学习（MARL）方法通常要求智能体具有全局状态可观测性，这阻碍了去中心化算法的开发并限制了可扩展性。最近的研究表明，在智能体间影响衰减的假设下，全局可观测性可以被每个智能体的局部邻域可观测性所取代，从而实现去中心化和可扩展性。然而，尽管由于路径损耗导致的信号功率衰减（即信号衰减）是无线通信和雷达网络中许多问题的固有特征，但具有此类衰减特性的实际应用仍未得到充分探索。在本文中，我们通过考虑在雷达网络中进行目标检测的功率分配这一说明性特例，证明了信号衰减使MARL中的去中心化成为可能。为此，我们针对该功率分配问题提出了两种新的受约束多智能体马尔可夫决策过程公式，推导了全局价值函数和策略梯度估计的局部邻域近似并建立了相应的误差界限，并开发了用于求解所提问题的去中心化鞍点策略梯度算法。我们的方法虽然面向我们所考虑的特定雷达网络问题，但为扩展到无线通信和雷达网络中的其他问题提供了有用的模型。
英文摘要：
Multi-agent reinforcement learning (MARL) methods typically require that agents enjoy global state observability, preventing development of decentralized algorithms and limiting scalability. Recent work has shown that, under assumptions on decaying inter-agent influence, global observability can be replaced by local neighborhood observability at each agent, enabling decentralization and scalability. Real-world applications enjoying such decay properties remain underexplored, however, despite the fact that signal power decay, or signal attenuation, due to path loss is an intrinsic feature of many problems in wireless communications and radar networks. In this paper, we show that signal attenuation enables decentralization in MARL by considering the illustrative special case of performing power allocation for target detection in a radar network. To achieve this, we propose two new constrained multi-agent Markov decision process formulations of this power allocation problem, derive local neighborhood approximations for global value function and policy gradient estimates and establish corresponding error bounds, and develop decentralized saddle point policy gradient algorithms for solving the proposed problems. Our approach, though oriented towards the specific radar network problem we consider, provides a useful model for extensions to additional problems in wireless communications and radar networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
