AI practitioners typically strive to develop the most accurate systems, making an implicit assumption that the AI system will function autonomously. However, in practice, AI systems often are used to provide advice to people in domains ranging from criminal justice and finance to healthcare. In such AI-advised decision making, humans and machines form a team, where the human is responsible for making final decisions. But is the most accurate AI the best teammate? We argue "No" -- predictable performance may be worth a slight sacrifice in AI accuracy. Instead, we argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. To optimize the team performance for this setting we maximize the team's expected utility, expressed in terms of the quality of the final decision, cost of verifying, and individual accuracies of people and machines. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance and show the benefit of modeling teamwork during training through improvements in expected team utility across datasets, considering parameters such as human skill and the cost of mistakes. We discuss the shortcoming of current optimization approaches beyond well-studied loss functions such as log-loss, and encourage future work on AI optimization problems motivated by human-AI collaboration.
ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork
Department of Computer Science（计算机科学系）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；College of Computing and Data Science（计算与数据科学学院）；Nanyang Technological University（南洋理工大学）
Symmetry-Breaking Augmentations for Ad Hoc Teamwork
Foerster Lab for AI Research, University of Oxford（牛津大学人工智能研究实验室）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学人工智能研究所）；Meta AI Research, UK（英国Meta人工智能研究）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Making Friends in the Dark: Ad Hoc Teamwork Under Partial Observability
INESC-ID（INESC-ID（高等工程系统与计算机研究所））；Instituto Superior Técnico, University of Lisbon（里斯本大学高等技术学院）；Pontifícia Universidade Católica do Rio de Janeiro（里约热内卢天主教大学）
Bar Ilan University（巴伊兰大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；The University of Edinburgh（爱丁堡大学）；The University of Birmingham（伯明翰大学）；Sony AI（索尼人工智能）
Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
C$^{2}$DL, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所C²DL实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；University of Science and Technology of China（中国科学技术大学）；Qwen Team, Alibaba Group（阿里集团Qwen团队）
Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi
MIT Lincoln Laboratory（麻省理工学院林肯实验室）；MIT Department of Electrical Engineering and Computer Science（麻省理工学院电气工程与计算机科学系）；U.S. Air Force Artificial Intelligence Accelerator（美国空军人工智能加速器）
CoSLight: Co-optimizing Collaborator Selection and Decision-making to Enhance Traffic Signal Control
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Cologne EWI gGmbH（科隆大学EWI有限责任公司）；Arizona State University（亚利桑那州立大学）；Baidu Inc.（百度公司）；Fudan University（复旦大学）；Peking University（北京大学）；Qing Yuan Research Institute of Shanghai Jiao Tong University（上海交通大学清源研究院）
