人机协作流程在搬运、焊接和装配等多个工业应用中的日益普及，推动了对能够管理大规模异构团队并同时监控复杂任务执行情况的系统的追求。在本文中，我们提出了一种新颖的架构，用于在任意规模的混合人机团队中进行动态角色分配和协作任务规划。该架构利用基于行为树（Behavior Trees, BTs）的集中式反应式、模块化且任务无关的规划方法，负责动作调度；而分配问题则通过混合整数线性规划（Mixed-Integer Linear Program, MILP）来表述，动态地为团队中的智能体分配个体角色或协作任务。使用不同的指标作为MILP成本，使该架构能够侧重协作的不同方面（例如完工时间、人机工程学、人类偏好）。人类偏好通过一个协商阶段来识别，在该阶段中，人类智能体可以接受或拒绝执行所分配的任务。此外，人类与系统之间的双向通信通过一个增强现实（Augmented Reality, AR）定制用户界面实现，该界面提供直观的功能，以在不同的动作阶段协助和协调工人。所提出方法的计算复杂度优于文献中针对工业规模作业和团队的方法（在包含协作的情况下，最多50个动作和20个智能体的问题可在1秒内求解）。随着成本函数的变化，所分配的不同角色凸显了该架构对多种生产需求的灵活性。最后，主观评估证明了其高可用性水平以及对目标场景的适用性。
英文摘要：
The growing deployment of human-robot collaborative processes in several industrial applications, such as handling, welding, and assembly, unfolds the pursuit of systems which are able to manage large heterogeneous teams and, at the same time, monitor the execution of complex tasks. In this paper, we present a novel architecture for dynamic role allocation and collaborative task planning in a mixed human-robot team of arbitrary size. The architecture capitalizes on a centralized reactive and modular task-agnostic planning method based on Behavior Trees (BTs), in charge of actions scheduling, while the allocation problem is formulated through a Mixed-Integer Linear Program (MILP), that assigns dynamically individual roles or collaborations to the agents of the team. Different metrics used as MILP cost allow the architecture to favor various aspects of the collaboration (e.g. makespan, ergonomics, human preferences). Human preference are identified through a negotiation phase, in which, an human agent can accept/refuse to execute the assigned task.In addition, bilateral communication between humans and the system is achieved through an Augmented Reality (AR) custom user interface that provides intuitive functionalities to assist and coordinate workers in different action phases. The computational complexity of the proposed methodology outperforms literature approaches in industrial sized jobs and teams (problems up to 50 actions and 20 agents in the team with collaborations are solved within 1 s). The different allocated roles, as the cost functions change, highlights the flexibility of the architecture to several production requirements. Finally, the subjective evaluation demonstrating the high usability level and the suitability for the targeted scenario.
In human-robot teams where agents collaborate together, there needs to be a clear allocation of tasks to agents. Task allocation can aid in achieving the presumed benefits of human-robot teams, such as improved team performance. Many task allocation methods have been proposed that include factors such as agent capability, availability, workload, fatigue, and task and domain-specific parameters. In this paper, selected work on task allocation is reviewed. In addition, some areas for continued and further consideration in task allocation are discussed. These areas include level of collaboration,…
UNSW Sydney（新南威尔士大学悉尼分校）；Data61, CSIRO（联邦科学与工业研究组织Data61）；Defence Science and Technology Group（国防科学与技术集团）
Task allocation using a team or coalition of robots is one of the most important problems in robotics, computer science, operational research, and artificial intelligence. In recent work, research has focused on handling complex objectives and feasibility constraints amongst other variations of the multi-robot task allocation problem. There are many examples of important research progress in these directions. We present a general formulation of the task allocation problem that generalizes several versions that are well-studied. Our formulation includes the states of robots, tasks, and the surr…
University of Michigan（密歇根大学）；Stevens Institute of Technology（史蒂文斯理工学院）；US Army CCDC Ground Vehicle Systems Center（美国陆军作战能力发展司令部地面车辆系统中心）
In operations of multi-agent teams ranging from homogeneous robot swarms to heterogeneous human-autonomy teams, unexpected events might occur. While efficiency of operation for multi-agent task allocation problems is the primary objective, it is essential that the decision-making framework is intelligent enough to manage unexpected task load with limited resources. Otherwise, operation effectiveness would drastically plummet with overloaded agents facing unforeseen risks. In this work, we present a decision-making framework for multi-agent teams to learn task allocation with the consideration …
Human-robot teams have the ability to perform better across various tasks than human-only and robot-only teams. However, such improvements cannot be realized without proper task allocation. Trust is an important factor in teaming relationships, and can be used in the task allocation strategy. Despite the importance, most existing task allocation strategies do not incorporate trust. This paper reviews select studies on trust and task allocation. We also summarize and discuss how a bi-directional trust model can be used for a task allocation strategy. The bi-directional trust model represents ta…
Consider a dynamic task allocation problem, where tasks are unknowingly distributed over an environment. This paper considers each task comprised of two sequential subtasks: detection and completion, where each subtask can only be carried out by a certain type of agent. We address this problem using a novel nature-inspired approach called "hunter and gatherer". The proposed method employs two complementary teams of agents: one agile in detecting (hunters) and another skillful in completing (gatherers) the tasks. To minimize the collective cost of task accomplishments in a distributed manner, a…
Shared Object Manipulation with a Team of Collaborative Quadrupeds
Department of Mechanical and Automation Engineering at The Chinese University of Hong Kong（香港中文大学机械与自动化工程系）；Technology and Innovation Center (TIC), KUKA Deutschland GmbH（KUKA德国技术与创新中心）
Recent works explore collaboration between humans and teams of robots. These approaches make sense if the human is already working with the robot team; but how should robots encourage nearby humans to join their teams in the first place? Inspired by behavioral economics, we recognize that humans care about more than just team efficiency -- humans also have biases and expectations for team dynamics. Our hypothesis is that the way inclusive robots divide the task (i.e., how the robots split a larger task into subtask allocations) should be both legible and fair to the human partner. In this pape…
This article presents an approximation algorithm for a task allocation, sequencing and scheduling problem involving a team of human operators and robots. Specifically, we present an algorithm with an approximation ratio as a function of the number of human operators ($m$) and the number of robots ($k$) in the team. The approximation ratios are $\frac{7}{2} -\frac{5}{2k}$, $\frac{5}{2} -\frac{1}{k}$ and $\frac{7}{2} -\frac{1}{k}$ when $m=1$, $m\geq k\geq 2$ and $k>m\geq 2$ respectively. We also present computational results to corroborate the performance of the proposed approximation algorithm.
Istituto Italiano di Tecnologia（意大利技术研究院）；University of Genoa（热那亚大学）；Politecnico di Milano（米兰理工大学）
By incorporating ergonomics principles into the task allocation processes, human-robot collaboration (HRC) frameworks can favour the prevention of work-related musculoskeletal disorders (WMSDs). In this context, existing offline methodologies do not account for the variability of human actions and states; therefore, planning and dynamically assigning roles in human-robot teams remains an unaddressed challenge.This study aims to create an ergonomic role allocation framework that optimises the HRC, taking into account task features and human state measurements. The presented framework consists o…
We study real-time collaborative robot (cobot) handling, where the cobot maneuvers a workpiece under human commands. This is useful when it is risky for humans to directly handle the workpiece. However, it is hard to make the cobot both easy to command and flexible in possible operations. In this work, we propose a Real-Time Collaborative Robot Handling (RTCoHand) framework that allows the control of cobot via user-customized dynamic gestures. This is hard due to variations among users, human motion uncertainties, and noisy human input. We model the task as a probabilistic generative process, …
Southern University of Denmark（丹麦南方大学）；Delft University of Technology（代尔夫特理工大学）
We propose a combined method for the collaborative transportation of a suspended payload by a team of rotorcraft. A recent distance-based formation-motion control algorithm based on assigning distance disagreements among robots generates the acceleration signals to be tracked by the vehicles. In particular, the proposed method does not need global positions nor tracking prescribed trajectories for the motion of the members of the team. The acceleration signals are followed accurately by an Incremental Nonlinear Dynamic Inversion controller designed for rotorcraft that measures and resists the …
Energy-Aware Collaborative Exploration for a UAV-UGV Team
Department of Mechanical and Industrial Engineering at Northeastern University（东北大学机械与工业工程系）；Departments of Mechanical and Industrial Engineering and Electrical and Computer Engineering at Northeastern University（东北大学机械与工业工程系和电气与计算机工程系）
University of Zagreb（萨格勒布大学）；Universidad de Sevilla（塞维利亚大学）
To enable safe and efficient use of multi-robot systems in everyday life, a robust and fast method for coordinating their actions must be developed. In this paper, we present a distributed task allocation and scheduling algorithm for missions where the tasks of different robots are tightly coupled with temporal and precedence constraints. The approach is based on representing the problem as a variant of the vehicle routing problem, and the solution is found using a distributed metaheuristic algorithm based on evolutionary computation (CBM-pop). Such an approach allows a fast and near-optimal a…
Task allocation in heterogeneous multi-agent teams often requires reasoning about multi-dimensional agent traits (i.e., capabilities) and the demands placed on them by tasks. However, existing methods tend to ignore the fact that not all traits equally contribute to a given task. Ignoring such inherent preferences or relative importance can lead to unintended sub-optimal allocations of limited agent resources that do not necessarily contribute to task success. Further, reasoning over a large number of traits can incur a hefty computational burden. To alleviate these concerns, we propose an alg…
Roma Tre University（罗马第三大学）；University of Cassino and Southern Lazio（卡西诺和南拉齐奥大学）
The requirements of modern production systems together with more advanced robotic technologies have fostered the integration of teams comprising humans and autonomous robots. However, along with the potential benefits also comes the question of how to effectively handle these teams considering the different characteristics of the involved agents. For this reason, this paper presents a framework for task allocation in a human multi-robot collaborative scenario. The proposed solution combines an optimal offline allocation with an online reallocation strategy which accounts for inaccuracies of th…
In this paper, we present a cyber-physical testbed created to enable a human-robot team to perform a shared task in a shared workspace. The testbed is suitable for the implementation of a tabletop manipulation task, a common human-robot collaboration scenario. The testbed integrates elements that exist in the physical and virtual world. In this work, we report the insights we gathered throughout our exploration in understanding and implementing task planning and execution for human-robot team.
Georgia Institute of Technology（佐治亚理工学院）；Optimus Ride
Multi-robot task allocation is a ubiquitous problem in robotics due to its applicability in a variety of scenarios. Adaptive task-allocation algorithms account for unknown disturbances and unpredicted phenomena in the environment where robots are deployed to execute tasks. However, this adaptivity typically comes at the cost of requiring precise knowledge of robot models in order to evaluate the allocation effectiveness and to adjust the task assignment online. As such, environmental disturbances can significantly degrade the accuracy of the models which in turn negatively affects the quality …
For a team of heterogeneous robots executing multiple tasks, we propose a novel algorithm to optimally allocate tasks to robots while accounting for their different capabilities. Motivated by the need that robot teams have in many real-world applications of remaining operational for long periods of time, we allow each robot to choose tasks taking into account the energy consumed by executing them, besides the global specifications on the task allocation. The tasks are encoded as constraints in an energy minimization problem solved at each point in time by each robot. The prioritization of a ta…
Collaboration between human supervisors and remote teams of robots is highly challenging, particularly in high-stakes, distant, hazardous locations, such as off-shore energy platforms. In order for these teams of robots to truly be beneficial, they need to be trusted to operate autonomously, performing tasks such as inspection and emergency response, thus reducing the number of personnel placed in harm's way. As remote robots are generally trusted less than robots in close-proximity, we present a solution to instil trust in the operator through a `mediator robot' that can exhibit social skills…
Collaborative Planning with Concurrent Synchronization for Operationally Constrained UAV-UGV Teams
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；North Carolina State University（北卡罗来纳州立大学）；U.S. Army DEVCOM Army Research Laboratory (ARL)（美国陆军 DEVCOM 军事研究实验室 (ARL)）
Human Experience and Agent Teamwork Lab（人类体验与智能体团队合作实验室）；Harvey Mudd College（哈维穆德学院）
Automated scheduling is potentially a very useful tool for facilitating efficient, intuitive interactions between a robot and a human teammate. However, a current gapin automated scheduling is that it is not well understood how to best represent the timing uncertainty that human teammates introduce. This paper attempts to address this gap by designing an online human-robot collaborative packaging game that we use to build a model of human timing uncertainty from a population of crowd-workers. We conclude that heavy-tailed distributions are the best models of human temporal uncertainty, with a …
Georgia Institute of Technology（佐治亚理工学院）；Georgia Tech Research Institute（佐治亚理工研究院）；UBTECH North America Research and Development Center Corporation（优必选北美研发中心有限公司）
This paper takes the first step towards a reactive, hierarchical multi-robot task allocation and planning framework given a global Linear Temporal Logic specification. The capabilities of both quadrupedal and wheeled robots are leveraged via a heterogeneous team to accomplish a variety of navigation and delivery tasks. However, when deployed in the real world, all robots can be susceptible to different types of disturbances, including but not limited to locomotion failures, human interventions, and obstructions from the environment. To address these disturbances, we propose task-level local an…
Effective human-robot collaboration requires informed anticipation. The robot must anticipate the human's actions, but also react quickly and intuitively when its predictions are wrong. The robot must plan its actions to account for the human's own plan, with the knowledge that the human's behavior will change based on what the robot actually does. This cyclical game of predicting a human's future actions and generating a corresponding motion plan is extremely difficult to model using standard techniques. In this work, we describe a novel Model Predictive Control (MPC)-based framework for find…
Human and robot partners increasingly need to work together to perform tasks as a team. Robots designed for such collaboration must reason about how their task-completion strategies interplay with the behavior and skills of their human team members as they coordinate on achieving joint goals. Our goal in this work is to develop a computational framework for robot adaptation to human partners in human-robot team collaborations. We first present an algorithm for autonomously recognizing available task-completion strategies by observing human-human teams performing a collaborative task. By transf…
Georgia Institute of Technology（佐治亚理工学院）；CCDC Army Research Laboratory（美国陆军研究实验室）
For multi-robot teams with heterogeneous capabilities, typical task allocation methods assign tasks to robots based on the suitability of the robots to perform certain tasks as well as the requirements of the task itself. However, in real-world deployments of robot teams, the suitability of a robot might be unknown prior to deployment, or might vary due to changing environmental conditions. This paper presents an adaptive task allocation and task execution framework which allows individual robots to prioritize among tasks while explicitly taking into account their efficacy at performing the ta…
Colorado School of Mines（科罗拉多矿业学院）；Toyota Motor North America（丰田汽车北美公司）
Collaborative localization is an essential capability for a team of robots such as connected vehicles to collaboratively estimate object locations from multiple perspectives with reliant cooperation. To enable collaborative localization, four key challenges must be addressed, including modeling complex relationships between observed objects, fusing observations from an arbitrary number of collaborating robots, quantifying localization uncertainty, and addressing latency of robot communications. In this paper, we introduce a novel approach that integrates uncertainty-aware spatiotemporal graph …
Multi-human multi-robot (MH-MR) systems have the ability to combine the potential advantages of robotic systems with those of having humans in the loop. Robotic systems contribute precision performance and long operation on repetitive tasks without tiring, while humans in the loop improve situational awareness and enhance decision-making abilities. A system's ability to adapt allocated workload to changing conditions and the performance of each individual (human and robot) during the mission is vital to maintaining overall system performance. Previous works from literature including market-bas…
Robots are envisioned to work alongside humans in applications ranging from in-home assistance to collaborative manufacturing. Research on human-robot collaboration (HRC) has helped develop various aspects of social intelligence necessary for robots to participate in effective, fluid collaborations with humans. However, HRC research has focused on dyadic, structured, and minimal collaborations between humans and robots that may not fully represent the large scale and emergent nature of more complex, unstructured collaborative activities. Thus, there remains a need for shared testbeds, datasets…
Role-Adaptive Collaborative Formation Planning for Team of Quadruped Robots in Cluttered Environments
Robotics and Artificial Intelligence Group, Department of Computer, Electrical and Space Engineering, Lulea University of Technology（机器人与人工智能小组，计算机、电气与航天工程系，卢莱亚理工大学）
