STLGame：对抗性多智能体系统中的信号时序逻辑博弈
STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems
AI总结：
本文提出 STLGame，将对抗多智能体 STL 任务建模为零和博弈，并用可微 STL 与 FSP 求解 Nash 均衡，在车辆和无人机基准上验证其鲁棒性。
摘要：
我们研究在对抗环境中，面对未知动态智能体时，如何为处于信号时序逻辑（STL）任务下的自主系统综合一种鲁棒且安全的策略。为确保最坏情况下的 STL 满足，我们提出了 STLGame，这是一个将多智能体系统建模为两人零和博弈的框架，其中自我智能体试图最大化 STL 满足程度，而其他智能体则试图将其最小化。STLGame 旨在利用虚拟自博弈（FSP）框架找到一个 Nash 均衡策略组合；就抵御未见过的对手策略的鲁棒性而言，这是最佳情形。即使在连续状态-动作空间中的博弈里，FSP 也会迭代收敛到 Nash 策略组合。我们提出了一种结合可微 STL 公式的基于梯度的方法，这对于在连续环境中于 FSP 的每次迭代中近似最佳响应至关重要。我们通过与用于寻找最佳响应的基于强化学习的方法进行比较，在实验上展示了这一关键方面。在两个标准动力系统基准（Ackermann 转向车辆和自主无人机）上的实验表明，我们收敛得到的策略几乎不可被利用，并且对各种未见过的对手策略具有鲁棒性。所有代码和额外实验结果均可在我们的项目网站 https://sites.google.com/view/stlgame 上找到。
英文摘要：
We study how to synthesize a robust and safe policy for autonomous systems under signal temporal logic (STL) tasks in adversarial settings against unknown dynamic agents. To ensure the worst-case STL satisfaction, we propose STLGame, a framework that models the multi-agent system as a two-player zero-sum game, where the ego agents try to maximize the STL satisfaction and other agents minimize it. STLGame aims to find a Nash equilibrium policy profile, which is the best case in terms of robustness against unseen opponent policies, by using the fictitious self-play (FSP) framework. FSP iteratively converges to a Nash profile, even in games set in continuous state-action spaces. We propose a gradient-based method with differentiable STL formulas, which is crucial in continuous settings to approximate the best responses at each iteration of FSP. We show this key aspect experimentally by comparing with reinforcement learning-based methods to find the best response. Experiments on two standard dynamical system benchmarks, Ackermann steering vehicles and autonomous drones, demonstrate that our converged policy is almost unexploitable and robust to various unseen opponents' policies. All code and additional experimental results can be found on our project website: https://sites.google.com/view/stlgame
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
