零样本协作在即兴团队中的通用策略改进与差额奖励
Zero-Shot Coordination in Ad Hoc Teams with Generalized Policy Improvement and Difference Rewards
AI总结：
本文提出了一种通用策略改进与差额奖励方法，用于实现不同团队间的高效零样本知识转移，通过三个模拟环境和真实机器人系统验证了其有效性。
摘要：
现实中的多智能体系统可能需要即兴协作，其中智能体必须与之前未见过的队友协调以在零样本方式下完成任务。先前工作通常基于对新队友的推断模型选择预训练策略或预训练一个对潜在队友鲁棒的单一策略。相反，我们提出利用所有预训练策略在零样本迁移设置中。我们将此问题形式化为即兴多智能体马尔可夫决策过程，并提出一个解决方案，利用两个关键思想，通用策略改进和差额奖励，实现不同团队间的高效有效知识转移。我们实证表明，我们的算法，通用策略改进用于即兴协作（GPAT），成功在三个模拟环境中实现了对新团队的零样本迁移：协作觅食、捕食者-猎物和Overcooked。我们还在真实世界多机器人设置中展示了我们的算法。
英文摘要：
Real-world multi-agent systems may require ad hoc teaming, where an agent must coordinate with other previously unseen teammates to solve a task in a zero-shot manner. Prior work often either selects a pretrained policy based on an inferred model of the new teammates or pretrains a single policy that is robust to potential teammates. Instead, we propose to leverage all pretrained policies in a zero-shot transfer setting. We formalize this problem as an ad hoc multi-agent Markov decision process and present a solution that uses two key ideas, generalized policy improvement and difference rewards, for efficient and effective knowledge transfer between different teams. We empirically demonstrate that our algorithm, Generalized Policy improvement for Ad hoc Teaming (GPAT), successfully enables zero-shot transfer to new teams in three simulated environments: cooperative foraging, predator-prey, and Overcooked. We also demonstrate our algorithm in a real-world multi-robot setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
