arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.17367cs.LG
收藏

R2PS: 在部分可观测性下最坏情况鲁棒的实时追捕策略

R2PS: Worst-Case Robust Real-Time Pursuit Strategies under Partial Observability

School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室)

Runyu Lu, Ruochuan Shi, Yuanheng Zhu, Dongbin Zhao

AI总结:

本文提出R2PS方法,通过动态规划和强化学习框架,在部分可观测环境下实现鲁棒的实时追捕策略,能泛化到未见图结构并优于现有方法。

摘要:

在追捕-逃避游戏中计算最坏情况鲁棒策略是耗时的,尤其当考虑现实因素如部分可观测性时。尽管对一般安全很重要,基于图的PEGs的实时追捕策略在追捕者仅对逃避者位置有不完全信息时目前缺失。虽然最先进的强化学习(RL)方法如平衡策略泛化(EPG)和Grasper为学习鲁棒的图神经网络(GNN)策略提供了指导,但它们仅限于完美信息场景,不考虑逃避者可能预测追捕者行动的情况。本文介绍了首个在部分可观测性下最坏情况鲁棒实时追捕策略(R2PS)的方法。我们首先证明传统动态规划(DP)算法在解决马尔可夫PEGs时在逃避者异步移动下保持最优。然后,我们提出关于逃避者可能位置的信念保持机制,将DP追捕策略扩展到部分可观测设置。最后,我们将信念保持嵌入到最先进的EPG框架中,完成我们的R2PS学习方案,通过跨图强化学习生成实时追捕者策略,对抗异步移动DP逃避策略。强化学习后,我们的策略在未见过的现实图结构上实现了鲁棒零样本泛化,并且在现有游戏RL方法直接训练在测试图上的策略上表现一致更优。

英文摘要:

Computing worst-case robust strategies in pursuit-evasion games (PEGs) is time-consuming, especially when real-world factors like partial observability are considered. While important for general security purposes, real-time applicable pursuit strategies for graph-based PEGs are currently missing when the pursuers only have imperfect information about the evader's position. Although state-of-the-art reinforcement learning (RL) methods like Equilibrium Policy Generalization (EPG) and Grasper provide guidelines for learning graph neural network (GNN) policies robust to different game dynamics, they are restricted to the scenario of perfect information and do not take into account the possible case where the evader can predict the pursuers' actions. This paper introduces the first approach to worst-case robust real-time pursuit strategies (R2PS) under partial observability. We first prove that a traditional dynamic programming (DP) algorithm for solving Markov PEGs maintains optimality under the asynchronous moves by the evader. Then, we propose a belief preservation mechanism about the evader's possible positions, extending the DP pursuit strategies to a partially observable setting. Finally, we embed the belief preservation into the state-of-the-art EPG framework to finish our R2PS learning scheme, which leads to a real-time pursuer policy through cross-graph reinforcement learning against the asynchronous-move DP evasion strategies. After reinforcement learning, our policy achieves robust zero-shot generalization to unseen real-world graph structures and consistently outperforms the policy directly trained on the test graphs by the existing game RL approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑