AI is becoming increasingly integrated into everyday life, both in professional work environments and in leisure and entertainment contexts. This integration requires AI to move beyond acting as an assistant for informational or transactional tasks toward a genuine collaborative partner. Effective collaboration, whether between humans or between humans and AI, depends on establishing and maintaining common ground: shared beliefs, assumptions, goals, and situational awareness that enable coordinated action and efficient repair of misunderstandings. While common ground is a central concept in human collaboration, it has received limited attention in studies of human-AI collaboration. In this paper, we introduce a new benchmark grounded in theories and empirical studies of human-human collaboration. The benchmark is based on a collaborative puzzle task that requires iterative interaction, joint action, referential coordination, and repair under varying conditions of situation awareness. We validate the benchmark through a confirmatory user study in which human participants collaborate with an AI to solve the task. The results show that the benchmark reproduces established theoretical and empirical findings from human-human collaboration, while also revealing clear divergences in human-AI interaction.
From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents
Northeastern University(东北大学);University of Notre Dame(圣母大学);ETH Zurich(苏黎世联邦理工学院);Carnegie Mellon University(卡内基梅隆大学)
Bidirectional human-AI collaboration in brain tumour assessments improves both expert human and AI agent performance
University College London(伦敦大学学院);National Hospital for Neurology and Neurosurgery(国立神经病学和神经外科医院);NVIDIA(英伟达);Great Ormond Street Hospital for Children(大奥蒙德街儿童医院);Royal National Orthopaedic Hospital(皇家国立骨科医院);University College Hospitals NHS Foundation Trust(伦敦大学学院医院国民保健服务基金会信托);Royal Free Hospital(皇家自由医院);Imperial College Healthcare NHS Trust(帝国理工学院医疗保健国民保健服务信托);Imperial College London(帝国理工学院)