arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.24992cs.NIcs.AIcs.LGcs.MA
收藏

面向任务驱动无人机网络的能量感知多智能体强化学习扩展与个体奖励

Scaling up Energy-Aware Multi-Agent Reinforcement Learning for Mission-Oriented Drone Networks with Individual Reward

Department of Computer Science, ETH Zurich(苏黎世联邦理工学院计算机科学系);Department of Computer Science, Colby College(科尔比学院计算机科学系)

Changling Li, Ying Li

AI总结:

提出基于个体奖励函数的能量感知多智能体强化学习模型,利用深度Q网络解决无人机网络动态环境和电池容量限制下的轨迹规划问题,实验表明在任务密度高时成功率接近100%,且扩展性优于共享奖励模型。

摘要:

多智能体强化学习(MARL)因其通过交互学习的能力,在自动驾驶和智慧城市等协作系统中显示出广泛适用性。随着无人机网络的最新发展,研究人员也应用MARL来解决轨迹规划问题。然而,动态环境和有限的电池容量仍然是使用MARL实现高效协作任务执行的挑战。在本文中,我们提出了一种能量感知的MARL模型作为应对这些挑战的尝试,利用深度Q网络(DQN)和由任务执行进度及无人机剩余电量驱动的个体奖励函数。我们对所提出的模型进行了一系列仿真研究,并将其与共享奖励MARL进行比较,以探索MARL中信用分配的影响。结果表明,无论任务位置和长度如何,我们提出的模型都能达到至少80%的成功率。与共享奖励模式类似,个体奖励模式在任务密度高时可以获得更好的成功率,并且当任务密度接近40%时,几乎可以达到100%的成功率。我们提出的个体奖励模型的真正优势在环境扩展时得以显现。与共享奖励MARL的比较表明,我们提出的模型对环境大小和智能体数量的变化更加鲁棒。由于目标的清晰性,它可以用更少的步骤实现更高的成功率,从而更好地提高能源效率。

英文摘要:

Multi-agent reinforcement learning (MARL) has shown wide applicability in collaborative systems such as autonomous driving and smart cities for its ability of learning through interaction. With the recent development of drone networks, researchers have also applied MARL to address the trajectory planning problems. However, the dynamic environment and the limited battery capacity are still challenging for using MARL to achieve efficient collaborative task execution. In this paper, we propose an energy-aware MARL model as an attempt to tackle these challenges, leveraging Deep Q-Networks (DQN) with \emph{individual reward functions} driven by the task execution progress and the remaining battery of drones. We conduct a set of simulation studies for the proposed mode and compare it with the shared reward MARL~\cite{Li2022MARL} to explore the impact of credit assignment in MARL. The results indicate that our proposed model can achieve at least 80\% success rate regardless of the task locations and lengths. Similar to the shared reward mode, the individual reward mode can achieve a better success rate when the task density is high, and it can hit nearly a 100\% success rate when task density gets close to 40\%. The true advantage of our proposed model with individual reward is revealed when scaling up the environment. The comparison to the shared reward MARL shows that the our proposed model is more robust towards the change of the environment size and agent numbers. It can achieve higher success rate with fewer steps due to the clarity of the goal which improves energy efficiency even better.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑