A2C: A Modular Multi-stage Collaborative Decision Framework for Human-AI Teams
Institute for Clarity in Documentation（文献清晰度研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康库尔中心）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕尔默研究实验室）
Shahroz Tariq, Mohan Baruwal Chhetri, Surya Nepal, Cecile Paris
This paper introduces A2C, a multi-stage collaborative decision framework designed to enable robust decision-making within human-AI teams. Drawing inspiration from concepts such as rejection learning and learning to defer, A2C incorporates AI systems trained to recognise uncertainty in their decisions and defer to human experts when needed. Moreover, A2C caters to scenarios where even human experts encounter limitations, such as in incident detection and response in cyber Security Operations Centres (SOC). In such scenarios, A2C facilitates collaborative explorations, enabling collective resolution of complex challenges. With support for three distinct decision-making modes in human-AI teams: Automated, Augmented, and Collaborative, A2C offers a flexible platform for developing effective strategies for human-AI collaboration. By harnessing the strengths of both humans and AI, it significantly improves the efficiency and effectiveness of complex decision-making in dynamic and evolving environments. To validate A2C's capabilities, we conducted extensive simulative experiments using benchmark datasets. The results clearly demonstrate that all three modes of decision-making can be effectively supported by A2C. Most notably, collaborative exploration by (simulated) human experts and AI achieves superior performance compared to AI in isolation, underscoring the framework's potential to enhance decision-making within human-AI teams.
University of Delaware（特拉华大学）；Computer and Information Sciences（计算机与信息科学学院）
Automatic assessment and evaluation of team performance during collaborative tasks is key to the learning analytics and computer-supported cooperative work research. There is a growing interest in the use of gaze-oriented cues for evaluating the collaboration and cooperativeness of teams. However, collecting gaze data using eye-trackers is not always feasible due to time and cost constraints. In this paper, we introduce an automated team assessment tool based on gaze points and joint visual attention (JVA) information extracted by computer vision solutions. We then evaluate team collaborations…
The emergence of several trends, including the increased availability of wireless networks, miniaturization of electronics and sensing technologies, and novel input and output devices, is creating a demand for integrated, full-time displays for use across a wide range of applications, including collaborative environments. In this paper, we present and discuss emerging visualization methods we are developing particularly as they relate to deployable displays and displays worn on the body to support mobile users.
Study Protocol: Shared Achievements: Exploring the Design of Gameful Collaborative Elements and Fostering Social Relatedness through Team Effort Contributions in a Social Physical Activity App
Human and robot partners increasingly need to work together to perform tasks as a team. Robots designed for such collaboration must reason about how their task-completion strategies interplay with the behavior and skills of their human team members as they coordinate on achieving joint goals. Our goal in this work is to develop a computational framework for robot adaptation to human partners in human-robot team collaborations. We first present an algorithm for autonomously recognizing available task-completion strategies by observing human-human teams performing a collaborative task. By transf…
We study real-time collaborative robot (cobot) handling, where the cobot maneuvers a workpiece under human commands. This is useful when it is risky for humans to directly handle the workpiece. However, it is hard to make the cobot both easy to command and flexible in possible operations. In this work, we propose a Real-Time Collaborative Robot Handling (RTCoHand) framework that allows the control of cobot via user-customized dynamic gestures. This is hard due to variations among users, human motion uncertainties, and noisy human input. We model the task as a probabilistic generative process, …
In conventional software development, user experience (UX) designers and engineers collaborate through separation of concerns (SoC): designers create human interface specifications, and engineers build to those specifications. However, we argue that Human-AI systems thwart SoC because human needs must shape the design of the AI interface, the underlying AI sub-components, and training data. How do designers and engineers currently collaborate on AI and UX design? To find out, we interviewed 21 industry professionals (UX researchers, AI engineers, data scientists, and managers) across 14 organi…
Paul G. Allen School of Computer Science & Engineering（保罗·G·艾伦计算机科学与工程学院）；University of Washington（华盛顿大学）
We focus on the problem of designing an artificial agent (AI), capable of assisting a human user to complete a task. Our goal is to guide human users towards optimal task performance while keeping their cognitive load as low as possible. Our insight is that doing so requires an understanding of human decision making for the task domain at hand. In this work, we consider the domain of collaborative packing, in which an AI agent provides placement recommendations to a human user. As a first step, we explore the mechanisms underlying human packing strategies. We conducted a user study in which 10…
Collaboration between human supervisors and remote teams of robots is highly challenging, particularly in high-stakes, distant, hazardous locations, such as off-shore energy platforms. In order for these teams of robots to truly be beneficial, they need to be trusted to operate autonomously, performing tasks such as inspection and emergency response, thus reducing the number of personnel placed in harm's way. As remote robots are generally trusted less than robots in close-proximity, we present a solution to instil trust in the operator through a `mediator robot' that can exhibit social skills…
Social ties are the invisible glue that keeps together human ecosystems. Despite the massive amount of research studying the role of social ties in communities (groups, teams, etc.) and society at large, little attention has been devoted to study their interplay with other human behavioral dynamics. Of particular interest is the influence that social ties have on human performance in collaborative team-based settings. Our research aims to elucidate the influence of social ties on individual and team performance dynamics. We will focus on a popular Multiplayer Online Battle Arena (MOBA) collabo…
Despite the intricacies involved in designing a computer as a teampartner, we can observe patterns in team behavior which allow us to describe at a general level how AI systems are to collaborate with humans. Whereas most work on human-machine teaming has focused on physical agents (e.g. robotic systems), our aim is to study how humans can collaborate with information agents. We propose some appropriate team design patterns, and test them using our Collaborative Intelligence Analysis (CIA) tool.
Will Your Next Pair Programming Partner Be Human? An Empirical Evaluation of Generative AI as a Collaborative Teammate in a Semester-Long Classroom Setting
Taking a picture has been traditionally a one-persons task. In this paper we present a novel system that allows multiple mobile devices to work collaboratively in a synchronized fashion to capture a panorama of a highly dynamic scene, creating an entirely new photography experience that encourages social interactions and teamwork. Our system contains two components: a client app that runs on all participating devices, and a server program that monitors and communicates with each device. In a capturing session, the server collects in realtime the viewfinder images of all devices and stitches th…
EPFL（洛桑联邦理工学院）；Polytechnique Institute of Paris（巴黎理工学院）；Télécom Paris（巴黎电信学院）
A dialogue is successful when there is alignment between the speakers at different linguistic levels. In this work, we consider the dialogue occurring between interlocutors engaged in a collaborative learning task, where they are not only evaluated on how well they performed, but also on how much they learnt. The main contribution of this work is to propose new automatic measures to study alignment; focusing on verbal (lexical) alignment, and behavioral alignment (when an instruction given by one was followed with concrete actions by another). A second contribution of our work is to study how …
Robotic collectives are large groups (at least 50) of locally sensing and communicating robots that encompass characteristics of swarms and colonies, whose emergent behaviors accomplish complex tasks. Future human-collective teams will extend the ability of operators to monitor, respond, and make decisions in disaster response, search and rescue, and environmental monitoring problems. This manuscript evaluates two collective best-of-n decision models for enabling collectives to identify and choose the highest valued target from a finite set of n targets. Two challenges impede the future use of…
How important are friendships in determining success by individuals and teams in complex collaborative environments? By combining a novel data set containing the dynamics of millions of ad hoc teams from the popular multiplayer online first person shooter Halo: Reach with survey data on player demographics, play style, psychometrics and friendships derived from an anonymous online survey, we investigate the impact of friendship on collaborative and competitive performance. In addition to finding significant differences in player behavior across these variables, we find that friendships exert a…
