As electronics manufacturers continue to face pressure to increase production efficiency amid difficulties with supply chains and labour shortages, many printed circuit board assembly (PCBA) manufacturers have begun to invest in automation and technological innovations to remain competitive. One such method is to leverage artificial intelligence (AI) to greatly augment existing manufacturing processes. In this paper, we present the DarwinAI Visual Quality Inspection (DVQI) system, a hardware-integration artificial intelligence system for the automated inspection of printed circuit board assembly defects in an electronics manufacturing environment. The DVQI system enables multi-task inspection via minimal programming and setup for manufacturing engineers while improving cycle time relative to manual inspection. We also present a case study of the deployed DVQI system's performance and impact for a top electronics manufacturer.
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（达尔文人工智能公司）；Moog Inc.（穆格公司）
In electronics manufacturing, solder joint defects are a common problem affecting a variety of printed circuit board components. To identify and correct solder joint defects, the solder joints on a circuit board are typically inspected manually by trained human inspectors, which is a very time-consuming and error-prone process. To improve both inspection efficiency and accuracy, in this work we describe an explainable deep learning-based visual quality inspection system tailored for visual inspection of solder joints in electronics manufacturing environments. At the core of this system is an e…
SenseTime Research（商汤科技研究院）；JD.com Inc Qing Yuan Research Institute（京东集团清源研究院）；Shanghai Jiao Tong University（上海交通大学）
Visual anomaly detection plays a crucial role in not only manufacturing inspection to find defects of products during manufacturing processes, but also maintenance inspection to keep equipment in optimum working condition particularly outdoors. Due to the scarcity of the defective samples, unsupervised anomaly detection has attracted great attention in recent years. However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying cam…
Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Recent advances in the industrial inspection of textured surfaces-in the form of visual inspection-have made such inspections possible for efficient, flexible manufacturing systems. We propose an unsupervised feature memory rearrangement network (FMR-Net) to accurately detect various textural defects simultaneously. Consistent with mainstream methods, we adopt the idea of background reconstruction; however, we innovatively utilize artificial synthetic defects to enable the model to recognize anomalies, while traditional wisdom relies only on defect-free samples. First, we employ an encoding mo…
A critical aspect in the manufacturing process is the visual quality inspection of manufactured components for defects and flaws. Human-only visual inspection can be very time-consuming and laborious, and is a significant bottleneck especially for high-throughput manufacturing scenarios. Given significant advances in the field of deep learning, automated visual quality inspection can lead to highly efficient and reliable detection of defects and flaws during the manufacturing process. However, deep learning-driven visual inspection methods often necessitate significant computational resources,…
IBM China Research Lab（IBM中国研究院）；Shanghai Jiao Tong University（上海交通大学）
With the rapid growth of display devices, quality inspection via machine vision technology has become increasingly important for flat-panel displays (FPD) industry. This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. The system is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture. A hybrid application of conventional computer vision technique and the latest deep convolutional neural network (DCNN) leads to an integrated defec…
Light guide plates are essential optical components widely used in a diverse range of applications ranging from medical lighting fixtures to back-lit TV displays. In this work, we introduce a fully-integrated, high-throughput, high-performance deep learning-driven workflow for light guide plate surface visual quality inspection (VQI) tailored for real-world manufacturing environments. To enable automated VQI on the edge computing within the fully-integrated VQI system, a highly compact deep anti-aliased attention condenser neural network (which we name LightDefectNet) tailored specifically for…
Adapting OpenAI's CLIP Model for Few-Shot Image Inspection in Manufacturing Quality Control: An Expository Case Study with Multiple Application Examples
Farmer School of Business, Miami University（迈阿密大学法默商学院）；College of Arts and Sciences, University of Dayton（Dayton大学文理学院）；Department of Mechanical Engineering, Politecnico di Milano（米兰理工学院机械工程系）；Mathematics & Statistics, Helmut Schmidt University（海姆-施密特大学数学与统计学系）；College of Engineering, University of Georgia（佐治亚大学工程学院）；Amsterdam Business School, University of Amsterdam（阿姆斯特丹大学阿姆斯特丹商学院）
Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Qlector d.o.o.（Qlector有限责任公司）；Faculty of Electrical Engineering, University of Ljubljana（卢布尔雅那大学电气工程学院）；Philips Consumer Lifestyle BV（飞利浦消费生活方式公司）
Quality control is a crucial activity performed by manufacturing enterprises to ensure that their products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. This research compares three active learning approaches, having single and multiple oracles, to visual inspection. Six new metrics are proposed to assess the quality of calibrati…
Federal University of Tecnology - Paraná (UTFPR)（巴拉那联邦理工大学）
Automotive manufacturing assembly tasks are built upon visual inspections such as scratch identification on machined surfaces, part identification and selection, etc, which guarantee product and process quality. These tasks can be related to more than one type of vehicle that is produced within the same manufacturing line. Visual inspection was essentially human-led but has recently been supplemented by the artificial perception provided by computer vision systems (CVSs). Despite their relevance, the accuracy of CVSs varies accordingly to environmental settings such as lighting, enclosure and …
Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Costa Rica Institute of Technology（哥斯达黎加理工学院）；Gestalt Automation GmbH（Gestalt Automation有限公司）；Technical University Berlin（柏林工业大学）
University of Ljubljana（卢布尔雅那大学）；Jožef Stefan International Postgraduate School（约热夫·斯特凡国际研究生院）；Philips Consumer Lifestyle BV（飞利浦消费生活方式有限公司）；Qlector d.o.o.（Qlector有限责任公司）；Jožef Stefan Institute（约热夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing enterprises to ensure products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. In this research, we compare three active learning approaches and five machine learning algorithms applied to visual defect inspection with real-world data provided by Philips Consumer Lifes…
Similar and indeterminate defect detection of solar cell surface with heterogeneous texture and complex background is a challenge of solar cell manufacturing. The traditional manufacturing process relies on human eye detection which requires a large number of workers without a stable and good detection effect. In order to solve the problem, a visual defect detection method based on multi-spectral deep convolutional neural network (CNN) is designed in this paper. Firstly, a selected CNN model is established. By adjusting the depth and width of the model, the influence of model depth and kernel …
Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；University of Ljubljana, Faculty of Electrical Engineering（卢布尔雅那大学电气工程学院）；Philips Consumer Lifestyle BV（飞利浦消费生活有限公司）；Qlector d.o.o.（Qlector有限公司）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing companies to verify product conformance to the requirements and specifications. Standardized quality control ensures that all the products are evaluated under the same criteria. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing and provided greater data availability. Such data availability has spurred the development of artificial intelligence models, which allow higher degrees of automation and reduced bias when inspecting the products. Furthermore, the increased speed of inspection…
VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）；Zhongguancun Academy（中关村学院）；School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）；School of AI, Beijing Institute of Technology（北京理工大学人工智能学院）；Beijing Key Laboratory of Lightweight Intelligent System, Beijing Institute of Technology（北京理工大学北京市轻量化智能系统重点实验室）
Nanyang Technological University（南洋理工大学）；Delta Research Center（台达研究院）
Automated defect inspection is critical for effective and efficient maintenance, repair, and operations in advanced manufacturing. On the other hand, automated defect inspection is often constrained by the lack of defect samples, especially when we adopt deep neural networks for this task. This paper presents Defect-GAN, an automated defect synthesis network that generates realistic and diverse defect samples for training accurate and robust defect inspection networks. Defect-GAN learns through defacement and restoration processes, where the defacement generates defects on normal surface image…
Photovoltaic cells are electronic devices that convert light energy to electricity, forming the backbone of solar energy harvesting systems. An essential step in the manufacturing process for photovoltaic cells is visual quality inspection using electroluminescence imaging to identify defects such as cracks, finger interruptions, and broken cells. A big challenge faced by industry in photovoltaic cell visual inspection is the fact that it is currently done manually by human inspectors, which is extremely time consuming, laborious, and prone to human error. While deep learning approaches holds …
Multi-Level Feature Fusion for Continual Learning in Visual Quality Inspection
Technical University of Munich, Institute for Machine Tools（慕尼黑技术大学，机床研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎矿学院，PSL大学，数学形态学中心（CMM））
Michigan State University（密歇根州立大学）；Thermal Wave Imaging, Inc.（热波成像公司）
Manufacturers struggle to produce low-cost, robust and complex components at manufacturing lot-size one. Additive processes like Fused Filament Fabrication (FFF) inexpensively produce complex geometries, but defects limit viability in critical applications. We present an approach to high-accuracy, high-throughput and low-cost automated non-destructive testing (NDT) for FFF interlayer delamination using Flash Thermography (FT) data processed with Thermographic Signal Reconstruction (TSR) and Artificial Intelligence (AI). A Deep Neural Network (DNN) attains 95.4% per-pixel accuracy when differen…
ContextMix: A context-aware data augmentation method for industrial visual inspection systems
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Samsung Electro-Mechanics（三星电机）；LG AI Research（LG AI研究院）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；RPTU Kaiserslautern-Landau（凯泽斯劳滕-兰道大学）
Quality inspection applications in industry are required to move towards a zero-defect manufacturing scenario, withnon-destructive inspection and traceability of 100 % of produced parts. Developing robust fault detection and classification modelsfrom the start-up of the lines is challenging due to the difficulty in getting enough representative samples of the faulty patternsand the need to manually label them. This work presents a methodology to develop a robust inspection system, targeting thesepeculiarities, in the context of solar cell manufacturing. The methodology is divided into two phas…
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Israel Atomic Energy Commission（以色列原子能委员会）；Technion – Israel Institute of Technology（以色列理工学院）；Nuclear Research Center – Negev（内盖夫核研究中心）
The usage of electronic devices increases, and becomes predominant in most aspects of life. Surface Mount Technology (SMT) is the most common industrial method for manufacturing electric devices in which electrical components are mounted directly onto the surface of a Printed Circuit Board (PCB). Although the expansion of electronic devices affects our lives in a productive way, failures or defects in the manufacturing procedure of those devices might also be counterproductive and even harmful in some cases. It is therefore desired and sometimes crucial to ensure zero-defect quality in electro…
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；National Technical University of Athens（雅典国立技术大学）；Philips Consumer Lifestyle BV（飞利浦消费者生活方式公司）；Qlector d.o.o.（Qlector公司）
Quality control is a crucial activity performed by manufacturing companies to ensure their products conform to the requirements and specifications. The introduction of artificial intelligence models enables to automate the visual quality inspection, speeding up the inspection process and ensuring all products are evaluated under the same criteria. In this research, we compare supervised and unsupervised defect detection techniques and explore data augmentation techniques to mitigate the data imbalance in the context of automated visual inspection. Furthermore, we use Generative Adversarial Net…
Human in the AI loop via xAI and Active Learning for Visual Inspection
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Qlector d.o.o.（Qlector公司）；University of Applied Sciences and Arts of Southern Switzerland（瑞士南部应用科学与艺术大学）；Ubitech Ltd.（Ubitech有限公司）；University of Ljubljana（卢布尔雅那大学）；University of West Attica（西阿提卡大学）；University of Groningen（格罗宁根大学）
本文提出结合主动学习与可解释人工智能，将人类纳入视觉检测的 AI 回路，并综述工业 5.0 人机协作及 STAR 项目相关成果。
The life time of electronic circuits board are impacted by the voids present in soldering balls. The quality inspection of solder balls by detecting and measuring the void is important to improve the board yield issues in electronic circuits. In general, the inspection is carried out manually, based on 2D or 3D X-ray images. For high quality inspection, it is difficult to detect and measure voids accurately with high repeatability through the manual inspection and it is time consuming process. In need of high quality and fast inspection, various approaches were proposed for void detection. But…
Enhancing Glass Defect Detection with Diffusion Models: Addressing Imbalanced Datasets in Manufacturing Quality Control
Department of Applied Statistics & Operations Research (ASOR), Bowling Green State University, Bowling Green, OH, USA（应用统计与运筹学系，布罗金格林州立大学）；Data Science Department, Actual Reality Technologies, OH, USA（数据科学系，Actual Reality Technologies）
Real-Time Industrial Defect Detection on Edge Hardware Using Fine-Tuned YOLOv8: A Systematic Benchmark on the NEU Surface Defect Database and MVTec AD with Automotive & Battery Manufacturing Extensions
Southern University and A&M College（南方大学与农工学院）；Louisiana State University（路易斯安那州立大学）
Additive Manufacturing (AM) is a crucial component of the smart industry. In this paper, we propose an automated quality grading system for the AM process using a deep convolutional neural network (CNN) model. The CNN model is trained offline using the images of the internal and surface defects in the layer-by-layer deposition of materials and tested online by studying the performance of detecting and classifying the failure in AM process at different extruder speeds and temperatures. The model demonstrates the accuracy of 94% and specificity of 96%, as well as above 75% in three classifier me…
Light guide plates are essential optical components widely used in a diverse range of applications ranging from medical lighting fixtures to back-lit TV displays. An essential step in the manufacturing of light guide plates is the quality inspection of defects such as scratches, bright/dark spots, and impurities. This is mainly done in industry through manual visual inspection for plate pattern irregularities, which is time-consuming and prone to human error and thus act as a significant barrier to high-throughput production. Advances in deep learning-driven computer vision has led to the expl…
