In this paper, we present a novel Bayesian online prediction algorithm for the problem setting of ad hoc teamwork under partial observability (ATPO), which enables on-the-fly collaboration with unknown teammates performing an unknown task without needing a pre-coordination protocol. Unlike previous works that assume a fully observable state of the environment, ATPO accommodates partial observability, using the agent's observations to identify which task is being performed by the teammates. Our approach assumes neither that the teammate's actions are visible nor an environment reward signal. We evaluate ATPO in three domains -- two modified versions of the Pursuit domain with partial observability and the overcooked domain. Our results show that ATPO is effective and robust in identifying the teammate's task from a large library of possible tasks, efficient at solving it in near-optimal time, and scalable in adapting to increasingly larger problem sizes.
ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork
Department of Computer Science（计算机科学系）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；College of Computing and Data Science（计算与数据科学学院）；Nanyang Technological University（南洋理工大学）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Sony AI（索尼人工智能）
A desirable goal for autonomous agents is to be able to coordinate on the fly with previously unknown teammates. Known as "ad hoc teamwork", enabling such a capability has been receiving increasing attention in the research community. One of the central challenges in ad hoc teamwork is quickly recognizing the current plans of other agents and planning accordingly. In this paper, we focus on the scenario in which teammates can communicate with one another, but only at a cost. Thus, they must carefully balance plan recognition based on observations vs. that based on communication. This paper pro…
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）；Allen Institute for AI（艾伦人工智能研究所）
AI practitioners typically strive to develop the most accurate systems, making an implicit assumption that the AI system will function autonomously. However, in practice, AI systems often are used to provide advice to people in domains ranging from criminal justice and finance to healthcare. In such AI-advised decision making, humans and machines form a team, where the human is responsible for making final decisions. But is the most accurate AI the best teammate? We argue "No" -- predictable performance may be worth a slight sacrifice in AI accuracy. Instead, we argue that AI systems should be…
Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability
INESC-ID（INESC-ID（高等工程系统与计算机研究所））；Instituto Superior Técnico, University of Lisbon（里斯本大学高等技术学院）；Pontifícia Universidade Católica do Rio de Janeiro（里约热内卢天主教大学）
Bar Ilan University（巴伊兰大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；The University of Edinburgh（爱丁堡大学）；The University of Birmingham（伯明翰大学）；Sony AI（索尼人工智能）
Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution: First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problems that need to be addressed in ad hoc teamwork.
We propose the novel few-shot teamwork (FST) problem, where skilled agents trained in a team to complete one task are combined with skilled agents from different tasks, and together must learn to adapt to an unseen but related task. We discuss how the FST problem can be seen as addressing two separate problems: one of reducing the experience required to train a team of agents to complete a complex task; and one of collaborating with unfamiliar teammates to complete a new task. Progress towards solving FST could lead to progress in both multi-agent reinforcement learning and ad hoc teamwork.
Symmetry-Breaking Augmentations for Ad Hoc Teamwork
Foerster Lab for AI Research, University of Oxford（牛津大学人工智能研究实验室）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学人工智能研究所）；Meta AI Research, UK（英国Meta人工智能研究）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
C$^{2}$DL, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所C²DL实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；University of Science and Technology of China（中国科学技术大学）；Qwen Team, Alibaba Group（阿里集团Qwen团队）
