For planning an assembly of a product from a given set of parts, robots necessitate certain cognitive skills: high-level planning is needed to decide the order of actuation actions, while geometric reasoning is needed to check the feasibility of these actions. For collaborative assembly tasks with humans, robots require further cognitive capabilities, such as commonsense reasoning, sensing, and communication skills, not only to cope with the uncertainty caused by incomplete knowledge about the humans' behaviors but also to ensure safer collaborations. We propose a novel method for collaborative assembly planning under uncertainty, that utilizes hybrid conditional planning extended with commonsense reasoning and a rich set of communication actions for collaborative tasks. Our method is based on answer set programming. We show the applicability of our approach in a real-world assembly domain, where a bi-manual Baxter robot collaborates with a human teammate to assemble furniture. This manuscript is under consideration for acceptance in TPLP.
Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
Robots have been steadily increasing their presence in our daily lives, where they can work along with humans to provide assistance in various tasks on industry floors, in offices, and in homes. Automated assembly is one of the key applications of robots, and the next generation assembly systems could become much more efficient by creating collaborative human-robot systems. However, although collaborative robots have been around for decades, their application in truly collaborative systems has been limited. This is because a truly collaborative human-robot system needs to adjust its operation …
Planning is hard. The use of subgoals can make planning more tractable, but selecting these subgoals is computationally costly. What algorithms might enable us to reap the benefits of planning using subgoals while minimizing the computational overhead of selecting them? We propose visual scoping, a strategy that interleaves planning and acting by alternately defining a spatial region as the next subgoal and selecting actions to achieve it. We evaluated our visual scoping algorithm on a variety of physical assembly problems against two baselines: planning all subgoals in advance and planning wi…
University of California - San Diego（加利福尼亚大学圣迭戈分校）
The World Robotics Challenge (2018 & 2020) was designed to challenge teams to design systems that are easy to adapt to new tasks and to ensure robust operation in a semi-structured environment. We present a layered strategy to transform missions into tasks and actions and provide a set of strategies to address simple and complex failures. We propose a model for characterizing failures using this model and discuss repairs. Simple failures are by far the most common in our WRC system and we also present how we repaired them.
Precise coordinated planning over a forward time window enables safe and highly efficient motion when many robots must work together in tight spaces, but this would normally require centralised control of all devices which is difficult to scale. We demonstrate GBP Planning, a new purely distributed technique based on Gaussian Belief Propagation for multi-robot planning problems, formulated by a generic factor graph defining dynamics and collision constraints over a forward time window. In simulations, we show that our method allows high performance collaborative planning where robots are able …
Massachusetts Institute of Technology（麻省理工学院）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）
Multi-robot assembly systems are becoming increasingly appealing in manufacturing due to their ability to automatically, flexibly, and quickly construct desired structural designs. However, effectively planning for these systems in a manner that ensures each robot is simultaneously productive, and not idle, is challenging due to (1) the close proximity that the robots must operate in to manipulate the structure and (2) the inherent structural partial orderings on when each part can be installed. In this paper, we present a task and motion planning framework that jointly plans safe, low-makespa…
We study the problem of sequential task assignment and collision-free routing for large teams of robots in applications with inter-task precedence constraints (e.g., task $A$ and task $B$ must both be completed before task $C$ may begin). Such problems commonly occur in assembly planning for robotic manufacturing applications, in which sub-assemblies must be completed before they can be combined to form the final product. We propose a hierarchical algorithm for computing makespan-optimal solutions to the problem. The algorithm is evaluated on a set of randomly generated problem instances where…
RoCo Challenge at AAAI 2026: Benchmarking Robotic Collaborative Manipulation for Assembly Towards Industrial Automation
Nanyang Technological University, Singapore（南洋理工大学，新加坡）；Beijing University of Posts and Telecommunications, Beijing, China（北京邮电大学，北京，中国）；Carnegie Mellon University, Pittsburgh, USA（卡内基梅隆大学，匹兹堡，美国）；Agency for Science, Technology and Research (A*STAR), Singapore（科技研究局（A*STAR），新加坡）
PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
Mohamed bin Zayed University of Artificial Intelligence（莫扎德·本·扎耶德人工智能大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院）；Shenzhen Campus of Sun Yat-sen University（孙中山大学深圳校区）；Spatialtemporal AI（时空人工智能）；Shanghai Jiao Tong University（上海交通大学）
INRIA（法国国家信息与自动化研究所）；École normale supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；New York University（纽约大学）
This paper addresses the problem of copying an unknown assembly of primitives with known shape and appearance using information extracted from a single photograph by an off-the-shelf procedure for object detection and pose estimation. The proposed algorithm uses a simple combination of physical stability constraints, convex optimization and Monte Carlo tree search to plan assemblies as sequences of pick-and-place operations represented by STRIPS operators. It is efficient and, most importantly, robust to the errors in object detection and pose estimation unavoidable in any real robotic system.…
Assembly sequence planning (ASP) is the essential process for modern manufacturing, proven to be NP-complete thus its effective and efficient solution has been a challenge for researchers in the field. In this paper, we present a graph-transformer based framework for the ASP problem which is trained and demonstrated on a self-collected ASP database. The ASP database contains a self-collected set of LEGO models. The LEGO model is abstracted to a heterogeneous graph structure after a thorough analysis of the original structure and feature extraction. The ground truth assembly sequence is first g…
Georgia Institute of Technology（佐治亚理工学院）；Monash University（莫纳什大学）；Honda Research Institute（本田研究院）
Driving is a social activity: drivers often indicate their intent to change lanes via motion cues. We consider mixed-autonomy traffic where a Human-driven Vehicle (HV) and an Autonomous Vehicle (AV) drive together. We propose a planning framework where the degree to which the AV considers the other agent's reward is controlled by a selfishness factor. We test our approach on a simulated two-lane highway where the AV and HV merge into each other's lanes. In a user study with 21 subjects and 6 different selfishness factors, we found that our planning approach was sound and that both agents had l…
Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordination
Computer Vision and Robotics Laboratory (VeRLab), Department of Computer Science, Universidade Federal de Minas Gerais（计算机视觉与机器人实验室（VeRLab）、计算机科学系、巴西联邦大学矿务格里斯大学）
SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly
School of Information Science and Engineering, Yunnan University, Kunming, China（信息科学与工程学院，云南大学，昆明，中国）；Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China（智能系统与计算云南重点实验室，昆明，中国）
In horizontal collaborations, carriers form coalitions in order to perform parts of their logistics operations jointly. By exchanging transportation requests among each other, they can operate more efficiently and in a more sustainable way. Collaborative vehicle routing has been extensively discussed in the literature. We identify three major streams of research: (i) centralized collaborative planning, (ii) decentralized planning without auctions, and (ii) auction-based decentralized planning. For each of them we give a structured overview on the state of knowledge and discuss future research …
National University of Singapore（新加坡国立大学）；MIT Computer Science and Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）；Singapore-MIT GAMBIT Game Lab（新加坡-麻省理工学院GAMBIT游戏实验室）
We apply decision theoretic techniques to construct non-player characters that are able to assist a human player in collaborative games. The method is based on solving Markov decision processes, which can be difficult when the game state is described by many variables. To scale to more complex games, the method allows decomposition of a game task into subtasks, each of which can be modelled by a Markov decision process. Intention recognition is used to infer the subtask that the human is currently performing, allowing the helper to assist the human in performing the correct task. Experiments s…
Safe and efficient collaboration among multiple robots in unstructured environments is increasingly critical in the era of Industry 4.0. However, achieving robust and autonomous collaboration among humans and other robots requires modern robotic systems to have effective proximity perception and reactive obstacle avoidance. In this paper, we propose a novel methodology for reactive whole-body obstacle avoidance that ensures conflict-free robot-robot interactions even in dynamic environment. Unlike existing approaches based on Jacobian-type, sampling based or geometric techniques, our methodolo…
University of Melbourne（墨尔本大学）；Department of Defence Science（国防科学部）
Intelligent agents powered by AI planning assist people in complex scenarios, such as managing teams of semi-autonomous vehicles. However, AI planning models may be incomplete, leading to plans that do not adequately meet the stated objectives, especially in unpredicted situations. Humans, who are apt at identifying and adapting to unusual situations, may be able to assist planning agents in these situations by encoding their knowledge into a planner at run-time. We investigate whether people can collaborate with agents by providing their knowledge to an agent using linear temporal logic (LTL)…
Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization
Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong (Shenzhen)（深圳大数据研究院，香港中文大学（深圳））；Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC), University of Macau（智能城市物联网国家重点实验室，澳门大学）；Manifold Tech Limited（曼陀科技有限公司）；School of Electrical Engineering and Telecommunications, the University of New South Wales（新南威尔士大学电子与电信工程学院）
Decision making in the Agriculture domain can be a complex task. The land area allocated to each crop should be fixed every season according to several parameters: prices, demand, harvesting periods, seeds, ground, season etc... The decision to make becomes more difficult when a group of farmers must fix the price and all parameters all together. Generally, optimization models are useful for farmers to find no dominated solutions, but it remains difficult if the farmers have to agree on one solution. We combine two approaches in order to support a group of farmers engaged in this kind of decis…
EUPHORIA: Efficient Universal Planning via Hybrid Optimization for Robust Industrial Robotic Assembly
National Taiwan University（国立台湾大学）；MoonShine Animation Studio（MoonShine动画工作室）；National Cheng Kung University（国立成功大学）；The University of British Columbia（不列颠哥伦比亚大学）；National Yang Ming Chiao Tung University（阳明交通大学）
University of Bremen（不来梅大学）；Stuttgart University of Applied Sciences（斯图加特应用技术大学）；University of Hagen（哈根大学）
The paper presents an ant colony optimization metaheuristic for collaborative planning. Collaborative planning is used to coordinate individual plans of self-interested decision makers with private information in order to increase the overall benefit of the coalition. The method consists of a new search graph based on encoded solutions. Distributed and private information is integrated via voting mechanisms and via a simple but effective collaborative local search procedure. The approach is applied to a distributed variant of the multi-level lot-sizing problem and evaluated by means of 352 ben…
