Recognizing errors in assembly and maintenance procedures is valuable for industrial applications, since it can increase worker efficiency and prevent unplanned down-time. Although assembly state recognition is gaining attention, none of the current works investigate assembly error localization. Therefore, we propose StateDiffNet, which localizes assembly errors based on detecting the differences between a (correct) intended assembly state and a test image from a similar viewpoint. StateDiffNet is trained on synthetically generated image pairs, providing full control over the type of meaningful change that should be detected. The proposed approach is the first to correctly localize assembly errors taken from real ego-centric video data for both states and error types that are never presented during training. Furthermore, the deployment of change detection to this industrial application provides valuable insights and considerations into the mechanisms of state-of-the-art change detection algorithms. The code and data generation pipeline are publicly available at: https://timschoonbeek.github.io/error_seg.
Bosch Rexroth AG（博世力士乐股份公司）；University of Applied Sciences Darmstadt（达姆施塔特应用科学大学）
This work presents the Industrial Hand Action Dataset V1, an industrial assembly dataset consisting of 12 classes with 459,180 images in the basic version and 2,295,900 images after spatial augmentation. Compared to other freely available datasets tested, it has an above-average duration and, in addition, meets the technical and legal requirements for industrial assembly lines. Furthermore, the dataset contains occlusions, hand-object interaction, and various fine-grained human hand actions for industrial assembly tasks that were not found in combination in examined datasets. The recorded grou…
Leveraging Foundation Model Automatic Data Augmentation Strategies and Skeletal Points for Hands Action Recognition in Industrial Assembly Lines
Faculty of Robot Science and Engineering, Northeastern University（东北大学机器人科学与工程学院）；Foshan Graduate School, Northeastern University（东北大学佛山研究生院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；The State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University（东北大学流程工业综合自动化国家重点实验室）
A vision-based framework for human behavior understanding in industrial assembly lines
Institute of Computer Science, Foundation for Research and Technology - Hellas（希腊研究与技术基金会计算机科学研究所）；School of Rural Surveying and Geoinformatics Engineering, National Technical University of Athens（雅典国立技术大学乡村测量与地理信息工程学院）；Institute of Communication and Computer Systems, National Technical University of Athens（雅典国立技术大学通信与计算机系统研究所）
AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects
Rutgers, The State University of New Jersey（新泽西罗格斯大学）；The Australian National University（澳大利亚国立大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔兰根-纽伦堡弗里德里希-亚历山大大学）；Mitsubishi Electric Research Laboratories (MERL)（三菱电机研究实验室）
SmartFactory-KL（凯泽斯劳滕智能工厂）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Detecting small objects in video streams of head-worn augmented reality devices in near real-time is a huge challenge: training data is typically scarce, the input video stream can be of limited quality, and small objects are notoriously hard to detect. In industrial scenarios, however, it is often possible to leverage contextual knowledge for the detection of small objects. Furthermore, CAD data of objects are typically available and can be used to generate synthetic training data. We describe a near real-time small object detection pipeline for egocentric perception in a manual assembly scen…
Group-Mix SAM: Lightweight Solution for Industrial Assembly Line Applications
Northeastern University（东北大学）；Faculty of Robot Science and Engineering, Northeastern University（东北大学机器人科学与工程学院）；Foshan Graduate School, Northeastern University（东北大学佛山研究生院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；The State Key Laboratory of Synthetical Automation for Process Industries, Northeastern University（东北大学流程工业综合自动化国家重点实验室）
Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challenges
Institute of Computer Science, Foundation for Research and Technology – Hellas（计算机科学研究所，希腊基础研究与技术机构）；Computer Science Department, University of Crete（克里特大学计算机科学系）
IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistants
Fraunhofer IPK（弗劳恩霍夫研究所）；Technical University of Berlin（柏林技术大学）；University of Tübingen（图宾根大学）；RWTH Aachen University（亚琛工业大学）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Most of industrial robotic assembly tasks today require fixed initial conditions for successful assembly. These constraints induce high production costs and low adaptability to new tasks. In this work we aim towards flexible and adaptable robotic assembly by using 3D CAD models for all parts to be assembled. We focus on a generic assembly task - the Siemens Innovation Challenge - in which a robot needs to assemble a gear-like mechanism with high precision into an operating system. To obtain the millimeter-accuracy required for this task and industrial settings alike, we use a depth camera moun…
