不确定信号博弈中统一分配巡逻的强化学习
Reinforcement Learning for Unified Allocation and Patrolling in Signaling Games with Uncertainty
摘要：
绿色安全游戏（GSGs）已成功用于保护渔业，森林和野生动物等宝贵资源。虽然现实世界的部署涉及资源分配和随后的协调巡逻与通信和实时，不确定的信息，以前的游戏模型没有完全解决这两个阶段同时进行。此外，采用现有的解决方案策略是困难的，因为它们不能很好地扩展更大，更复杂的游戏模型的变体。因此，我们首先提出了一种新的GSG模型，该模型结合了防御者分配，巡逻，实时无人机通知人类巡逻人员，以及无人机向攻击者发送警告信号。该模型进一步将不确定性纳入无人机和人类巡逻人员团队的实时决策中。其次，我们提出了一种基于强化学习的新型可扩展算法CombSGPO来计算该游戏模型的防御策略。CombSGPO在多维离散动作空间上执行策略搜索，以计算最适合防御者的最佳响应巡逻策略的分配策略，该策略是通过训练多智能体Deep Q-Network学习的。我们通过实验表明，CombSGPO收敛到更好的策略，比同类方法更具可扩展性。第三，我们提供了一个详细的分析的协调和信令行为学到的CombSGPO，显示防御资源和巡逻编队之间的信令和资源之间的通知组的形成。重要的是，我们发现策略信号出现在最终学习策略中。最后，我们进行实验，以评估这些策略下不同程度的不确定性。
英文摘要：
Green Security Games (GSGs) have been successfully used in the protection of valuable resources such as fisheries, forests and wildlife. While real-world deployment involves both resource allocation and subsequent coordinated patrolling with communication and real-time, uncertain information, previous game models do not fully address both of these stages simultaneously. Furthermore, adopting existing solution strategies is difficult since they do not scale well for larger, more complex variants of the game models. We therefore first propose a novel GSG model that combines defender allocation, patrolling, real-time drone notification to human patrollers, and drones sending warning signals to attackers. The model further incorporates uncertainty for real-time decision-making within a team of drones and human patrollers. Second, we present CombSGPO, a novel and scalable algorithm based on reinforcement learning, to compute a defender strategy for this game model. CombSGPO performs policy search over a multi-dimensional, discrete action space to compute an allocation strategy that is best suited to a best-response patrolling strategy for the defender, learnt by training a multi-agent Deep Q-Network. We show via experiments that CombSGPO converges to better strategies and is more scalable than comparable approaches. Third, we provide a detailed analysis of the coordination and signaling behavior learnt by CombSGPO, showing group formation between defender resources and patrolling formations based on signaling and notifications between resources. Importantly, we find that strategic signaling emerges in the final learnt strategy. Finally, we perform experiments to evaluate these strategies under different levels of uncertainty.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
