基于多评判器策略梯度优化的无人机协同多智能体强化学习
Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimization
AI总结：
针对多智能体强化学习在无人机协同中存在的误差反向传播困难、忽略协作的问题，提出含多价值估计网络与新型优势函数的多评判器策略优化架构，实现稳定训练与最优协同。
摘要：
无人机（UAV）技术的最新进展以及采购成本的下降，使得无人机编队在各类任务中的应用颇具吸引力。在农业、灾害管理、搜救行动、商业及军事应用中，使用无人机编队的优势源于其自主协作的能力。多智能体强化学习方法旨在优化基于神经网络的控制策略，例如性能最优的演员-评判器策略梯度算法，但这类方法难以有效反向传播不同奖励信号源的误差，且倾向于优先处理收益更高的信号，却忽略了协作以及对已习得相似性的利用。我们提出一种多评判器策略优化架构，该架构包含多个价值估计网络和一种新型优势函数，可优化随机演员策略网络，以实现智能体的最优协同。随后，我们在基于物理的强化学习环境中，将该算法应用于多项需要多架无人机协作的任务。我们的方法实现了稳定的策略网络更新，且随着智能体数量增加，奖励信号的变化趋势保持一致。最终得到的策略实现了最优协同，并满足避撞等约束条件。
英文摘要：
Recent technological progress in the development of Unmanned Aerial Vehicles (UAVs) together with decreasing acquisition costs make the application of drone fleets attractive for a wide variety of tasks. In agriculture, disaster management, search and rescue operations, commercial and military applications, the advantage of applying a fleet of drones originates from their ability to cooperate autonomously. Multi-Agent Reinforcement Learning approaches that aim to optimize a neural network based control policy, such as the best performing actor-critic policy gradient algorithms, struggle to effectively back-propagate errors of distinct rewards signal sources and tend to favor lucrative signals while neglecting coordination and exploitation of previously learned similarities. We propose a Multi-Critic Policy Optimization architecture with multiple value estimating networks and a novel advantage function that optimizes a stochastic actor policy network to achieve optimal coordination of agents. Consequently, we apply the algorithm to several tasks that require the collaboration of multiple drones in a physics-based reinforcement learning environment. Our approach achieves a stable policy network update and similarity in reward signal development for an increasing number of agents. The resulting policy achieves optimal coordination and compliance with constraints such as collision avoidance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
