Is it possible to learn policies for robotic assembly that can generalize to new objects? We explore this idea in the context of the kit assembly task. Since classic methods rely heavily on object pose estimation, they often struggle to generalize to new objects without 3D CAD models or task-specific training data. In this work, we propose to formulate the kit assembly task as a shape matching problem, where the goal is to learn a shape descriptor that establishes geometric correspondences between object surfaces and their target placement locations from visual input. This formulation enables the model to acquire a broader understanding of how shapes and surfaces fit together for assembly -- allowing it to generalize to new objects and kits. To obtain training data for our model, we present a self-supervised data-collection pipeline that obtains ground truth object-to-placement correspondences by disassembling complete kits. Our resulting real-world system, Form2Fit, learns effective pick and place strategies for assembling objects into a variety of kits -- achieving $90\%$ average success rates under different initial conditions (e.g. varying object and kit poses), $94\%$ success under new configurations of multiple kits, and over $86\%$ success with completely new objects and kits.
WorkBenchMark: A LEGO-Based Assembly Benchmark with an Assembly-by-Disassembly Baseline for the Smart Manufacturing League
Chair of Machine Learning and Reasoning (i6), RWTH Aachen University(亚琛工业大学机器学习与推理教席(i6));MASCOR Institute, FH Aachen University of Applied Science(亚琛应用技术大学MASCOR研究所)
REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly
Autonomous Systems Lab, Institute of Computer Technology, TU Wien(自主系统实验室,计算机技术学院,维也纳技术大学);Institute of Robotics and Mechatronics (DLR), German Aerospace Center(机器人与机电系统研究所(DLR),德国航空航天中心)
PrimitiveVLA: Learning Reusable Motion Primitives for Efficient and Generalizable Robotic Manipulation
State Key Lab of Processors, Institute of Computing Technology, CAS(处理器国家重点实验室,计算技术研究所,中国科学院);Jiangsu Key Laboratory of AI for Industries, Institute of AI for Industries, CAS(江苏人工智能工业重点实验室,人工智能工业研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Cambricon Technologies(寒武科技);Intelligent Software Research Center, Institute of Software, CAS(软件研究所智能软件研究中心,中国科学院);University of Science and Technology of China(中国科学技术大学)
9DTact: A Compact Vision-Based Tactile Sensor for Accurate 3D Shape Reconstruction and Generalizable 6D Force Estimation
Shanghai Qi Zhi Institute(上海期智研究院);Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Tsinghua University(清华大学);Huazhong University of Science and Technology(华中科技大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)