The defect detection task can be regarded as a realistic scenario of object detection in the computer vision field and it is widely used in the industrial field. Directly applying vanilla object detector to defect detection task can achieve promising results, while there still exists challenging issues that have not been solved. The first issue is the texture shift which means a trained defect detector model will be easily affected by unseen texture, and the second issue is partial visual confusion which indicates that a partial defect box is visually similar with a complete box. To tackle these two problems, we propose a Reference-based Defect Detection Network (RDDN). Specifically, we introduce template reference and context reference to against those two problems, respectively. Template reference can reduce the texture shift from image, feature or region levels, and encourage the detectors to focus more on the defective area as a result. We can use either well-aligned template images or the outputs of a pseudo template generator as template references in this work, and they are jointly trained with detectors by the supervision of normal samples. To solve the partial visual confusion issue, we propose to leverage the carried context information of context reference, which is the concentric bigger box of each region proposal, to perform more accurate region classification and regression. Experiments on two defect detection datasets demonstrate the effectiveness of our proposed approach.
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Israel Atomic Energy Commission（以色列原子能委员会）；Technion – Israel Institute of Technology（以色列理工学院）；Nuclear Research Center – Negev（内盖夫核研究中心）
The usage of electronic devices increases, and becomes predominant in most aspects of life. Surface Mount Technology (SMT) is the most common industrial method for manufacturing electric devices in which electrical components are mounted directly onto the surface of a Printed Circuit Board (PCB). Although the expansion of electronic devices affects our lives in a productive way, failures or defects in the manufacturing procedure of those devices might also be counterproductive and even harmful in some cases. It is therefore desired and sometimes crucial to ensure zero-defect quality in electro…
Visual defect detection is critical to ensure the quality of most products. However, the majority of small and medium-sized manufacturing enterprises still rely on tedious and error-prone human manual inspection. The main reasons include: 1) the existing automated visual defect detection systems require altering production assembly lines, which is time consuming and expensive 2) the existing systems require manually collecting defective samples and labeling them for a comparison-based algorithm or training a machine learning model. This introduces a heavy burden for small and medium-sized manu…
For the sake of recognizing and classifying textile defects, deep learning-based methods have been proposed and achieved remarkable success in single-label textile images. However, detecting multi-label defects in a textile image remains challenging due to the coexistence of multiple defects and small-size defects. To address these challenges, a multi-level, multi-attentional deep learning network (MLMA-Net) is proposed and built to 1) increase the feature representation ability to detect small-size defects; 2) generate a discriminative representation that maximizes the capability of attending…
With the wide applications of Unmanned Aerial Vehicle (UAV) in engineering such as the inspection of the electrical equipment from distance, the demands of efficient object detection algorithms for abundant images acquired by UAV have also been significantly increased in recent years. In this work, we study the performance of the region-based CNN for the electrical equipment defect detection by using the UAV images. In order to train the detection model, we collect a UAV images dataset composes of four classes of electrical equipment defects with thousands of annotated labels. Then, based on t…
Quality control is a fundamental component of many manufacturing processes, especially those involving casting or welding. However, manual quality control procedures are often time-consuming and error-prone. In order to meet the growing demand for high-quality products, the use of intelligent visual inspection systems is becoming essential in production lines. Recently, Convolutional Neural Networks (CNNs) have shown outstanding performance in both image classification and localization tasks. In this article, a system is proposed for the identification of casting defects in X-ray images, based…
Dongguan Polytechnic（东莞职业技术学院）；Huazhong University of Science and Technology（华中科技大学）
For the problems of low recognition rate and slow recognition speed of traditional detection methods in IC appearance defect detection, we propose an IC appearance defect detection algo-rithm IH-ViT. Our proposed model takes advantage of the respective strengths of CNN and ViT to acquire image features from both local and global aspects, and finally fuses the two features for decision making to determine the class of defects, thus obtaining better accuracy of IC defect recognition. To address the problem that IC appearance defects are mainly reflected in the dif-ferences in details, which are …
High-Performance Fine Defect Detection in Artificial Leather Using Dual Feature Pool Object Detection
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Chongqing University（重庆大学）；Inspur Yunzhou Industrial Internet Co., Ltd（浪潮云洲工业互联网有限公司）；Guoqi Zhimo (Chongqing) Technology Co., Ltd.（国汽智磨（重庆）科技有限公司）；Shenzhen University（深圳大学）
Beihang University（北京航空航天大学）；Hebei University of Technology（河北工业大学）
The multi-scale defect detection for photovoltaic (PV) cell electroluminescence (EL) images is a challenging task, due to the feature vanishing as network deepens. To address this problem, an attention-based top-down and bottom-up architecture is developed to accomplish multi-scale feature fusion. This architecture, called Bidirectional Attention Feature Pyramid Network (BAFPN), can make all layers of the pyramid share similar semantic features. In BAFPN, cosine similarity is employed to measure the importance of each pixel in the fused features. Furthermore, a novel object detector is propose…
Visual Information Laboratory, University of Bristol, UK（布里斯托尔大学视觉信息实验室）
As a data-driven method, the performance of deep convolutional neural networks (CNN) relies heavily on training data. The prediction results of traditional networks give a bias toward larger classes, which tend to be the background in the semantic segmentation task. This becomes a major problem for fault detection, where the targets appear very small on the images and vary in both types and sizes. In this paper we propose a new network architecture, DefectNet, that offers multi-class (including but not limited to) defect detection on highly-imbalanced datasets. DefectNet consists of two parall…
School of Computer Science, Yangtze University（长江大学计算机科学学院）；School of Electronics & Information, Yangtze University（长江大学电子信息学院）；School of Urban Construction, Yangtze University（长江大学城市建设学院）
Steel pipes are widely used in high-risk and high-pressure scenarios such as oil, chemical, natural gas, shale gas, etc. If there is some defect in steel pipes, it will lead to serious adverse consequences. Applying object detection in the field of deep learning to pipe weld defect detection and identification can effectively improve inspection efficiency and promote the development of industrial automation. Most predecessors used traditional computer vision methods applied to detect defects of steel pipe weld seams. However, traditional computer vision methods rely on prior knowledge and can …
Electrical and Electronics Engineering Institute（电气与电子工程学院）；University of the Philippines Diliman（菲律宾大学迪利曼分校）
The inclusion of Computer Vision and Deep Learning technologies in Agriculture aims to increase the harvest quality, and productivity of farmers. During postharvest, the export market and quality evaluation are affected by assorting of fruits and vegetables. In particular, apples are susceptible to a wide range of defects that can occur during harvesting or/and during the post-harvesting period. This paper aims to help farmers with post-harvest handling by exploring if recent computer vision and deep learning methods such as the YOLOv3 (Redmon & Farhadi (2018)) can help in detecting healthy ap…
School of Electrical and Electronic Engineering, North China Electric Power University（华北电力大学电气与电子工程学院）；School of Computer Science and Technology, North China Electric Power University（华北电力大学计算机科学与技术学院）
Bolts are the most numerous fasteners in transmission lines and are prone to losing their split pins. How to realize the automatic pin-missing defect detection for bolts in transmission lines so as to achieve timely and efficient trouble shooting is a difficult problem and the long-term research target of power systems. In this paper, an automatic detection model called Automatic Visual Shape Clustering Network (AVSCNet) for pin-missing defect is constructed. Firstly, an unsupervised clustering method for the visual shapes of bolts is proposed and applied to construct a defect detection model …
In this research work, we have demonstrated the application of Mask-RCNN (Regional Convolutional Neural Network), a deep-learning algorithm for computer vision and specifically object detection, to semiconductor defect inspection domain. Stochastic defect detection and classification during semiconductor manufacturing has grown to be a challenging task as we continuously shrink circuit pattern dimensions (e.g., for pitches less than 32 nm). Defect inspection and analysis by state-of-the-art optical and e-beam inspection tools is generally driven by some rule-based techniques, which in turn oft…
New production techniques have emerged that have made it possible to produce metal parts with more complex shapes, making the quality control process more difficult. This implies that the visual and superficial analysis has become even more inefficient. On top of that, it is also not possible to detect internal defects that these parts could have. The use of X-Ray images has made this process much easier, allowing not only to detect superficial defects in a much simpler way, but also to detect welding or casting defects that could represent a serious hazard for the physical integrity of the me…
Profactor GmbH（Profactor有限公司）；JKU Institute of Computational Perception（约翰内斯·开普勒大学计算感知研究所）
Automated fiber placement (AFP) is an advanced manufacturing technology that increases the rate of production of composite materials. At the same time, the need for adaptable and fast inline control methods of such parts raises. Existing inspection systems make use of handcrafted filter chains and feature detectors, tuned for a specific measurement methods by domain experts. These methods hardly scale to new defects or different measurement devices. In this paper, we propose to formulate AFP defect detection as an image segmentation problem that can be solved in an end-to-end fashion using art…
Automated Defect Detection for Mass-Produced Electronic Components Based on YOLO Object Detection Models
Department of Electrical Engineering and Graduate School of Engineering Science and Technology（电气工程系和工程科学与技术研究生学院）；National Yunlin University of Science and Technology（云林科技大学）；Research Center for Information Technology Innovation（信息技术创新研究中心）；Academia Sinica（学术院）；Institute of Communications Engineering（通讯工程研究所）；National Sun Yat-sen University（国立中山大学）
This work is addressing the problem of defect anomaly detection based on a clean reference image. Specifically, we focus on SEM semiconductor defects in addition to several natural image anomalies. There are well-known methods to create a simulation of an artificial reference image by its defect specimen. In this work, we introduce several applications for this capability, that the simulated reference is beneficial for improving their results. Among these defect detection methods are classic computer vision applied on difference-image, supervised deep-learning (DL) based on human labels, and u…
Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Visual anomaly detection plays a significant role in the development of industrial automatic product quality inspection. As a result of the utmost imbalance in the amount of normal and abnormal data, growing attention has been given to unsupervised methods for defect detection. Although existing reconstruction-based methods have been widely studied recently, establishing a robust reconstruction model for various textured surface defect detection remains a challenging task due to homogeneous and nonregular surface textures. In this paper, we propose a novel unsupervised reconstruction-based met…
Defect detection in the manufacturing industry is of utmost importance for product quality inspection. Recently, optical defect detection has been investigated as an anomaly detection using different deep learning methods. However, the recent works do not explore the use of point pattern features, such as SIFT for anomaly detection using the recently developed set-based methods. In this paper, we present an evaluation of different point pattern feature detectors and descriptors for defect detection application. The evaluation is performed within the random finite set framework. Handcrafted poi…
New Horizon College of Engineering（新视野工程学院）；Govt. College of Engineering, Chamarajanagar（查马拉贾纳加尔政府工程学院）；University of Mysore（迈索尔大学）
In this paper, we propose a machine vision algorithm for automatically detecting defects in patterned textures with the help of gradient space and its energy. Experiments on real fabric images with defects show that the proposed method can be used for automatic detection of fabric defects in textile industries.
This paper focuses on fruit defect detection and glare removal using morphological operations, Glare removal can be considered as an important preprocessing step as uneven lighting may introduce it in images, which hamper the results produced through segmentation by Gabor filters .The problem of glare in images is very pronounced sometimes due to the unusual reflectance from the camera sensor or stray light entering, this method counteracts this problem and makes the defect detection much more pronounced. Anisotropic diffusion is used for further smoothening of the images and removing the high…
Leibniz University Hannover（莱布尼茨汉诺威大学）；University of British Columbia（英属哥伦比亚大学）
In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flo…
To coupe with the difficulties in the process of inspection and classification of defects in Printed Circuit Board (PCB), other researchers have proposed many methods. However, few of them published their dataset before, which hindered the introduction and comparison of new methods. In this paper, we published a synthesized PCB dataset containing 1386 images with 6 kinds of defects for the use of detection, classification and registration tasks. Besides, we proposed a reference based method to inspect and trained an end-to-end convolutional neural network to classify the defects. Unlike conven…
In this paper, we present a Robust Completed Local Binary Pattern (RCLBP) framework for a surface defect detection task. Our approach uses a combination of Non-Local (NL) means filter with wavelet thresholding and Completed Local Binary Pattern (CLBP) to extract robust features which are fed into classifiers for surface defects detection. This paper combines three components: A denoising technique based on Non-Local (NL) means filter with wavelet thresholding is established to denoise the noisy image while preserving the textures and edges. Second, discriminative features are extracted using t…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Surface defect detection is an extremely crucial step to ensure the quality of industrial products. Nowadays, convolutional neural networks (CNNs) based on encoder-decoder architecture have achieved tremendous success in various defect detection tasks. However, due to the intrinsic locality of convolution, they commonly exhibit a limitation in explicitly modeling long-range interactions, critical for pixel-wise defect detection in complex cases, e.g., cluttered background and illegible pseudo-defects. Recent transformers are especially skilled at learning global image dependencies but with lim…
Central China Normal University（华中师范大学）；Northwest Normal University（西北师范大学）
Photovoltaic (PV) panel surface-defect detection technology is crucial for the PV industry to perform smart maintenance. Using computer vision technology to detect PV panel surface defects can ensure better accuracy while reducing the workload of traditional worker field inspections. However, multiple tiny defects on the PV panel surface and the high similarity between different defects make it challenging to {accurately identify and detect such defects}. This paper proposes an approach named Ghost convolution with BottleneckCSP and a tiny target prediction head incorporating YOLOv5 (GBH-YOLOv…
New Horizon College of Engineering（新视野工程学院）；Govt. College of Engineering（政府工程学院）；University of Mysore（迈索尔大学）
In this paper, we propose a machine vision algorithm for automatically detecting defects in textures belonging to 16 out of 17 wallpaper groups using texture-periodicity and a family of Gabor wavelets. Input defective images are subjected to Gabor wavelet transformation in multi-scales and multi-orientations and a resultant image is obtained in L2 norm. The resultant image is split into several periodic blocks and energy of each block is used as a feature space to automatically identify defective and defect-free blocks using Ward's hierarchical clustering. Experiments on defective fabric image…
In industrial fabric productions, automated real time systems are needed to find out the minor defects. It will save the cost by not transporting defected products and also would help in making compmay image of quality fabrics by sending out only undefected products. A real time fabric defect detection system (FDDS), implementd on an embedded DSP platform is presented here. Textural features of fabric image are extracted based on gray level co-occurrence matrix (GLCM). A sliding window technique is used for defect detection where window moves over the whole image computing a textural energy fr…
A Comprehensive Survey for Real-World Industrial Defect Detection: Challenges, Approaches, and Prospects
State Key Laboratory of Intelligent Manufacturing Equipment and Technology, Huazhong University of Science and Technology（智能制造装备与技术国家重点实验室，华中科技大学）；School of Artificial Intelligence and Robotics, Hunan University（人工智能与机器人学院，湖南大学）；Department of Precision Instrument, Tsinghua University（精密仪器系，清华大学）；Geely Automobile Research Institute (Ningbo) Co., Ltd.（吉利汽车研究院（宁波）有限公司）
Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
Deep learning methods have proven to outperform traditional computer vision methods in various areas of image processing. However, the application of deep learning in industrial surface defect detection systems is challenging due to the insufficient amount of training data, the expensive data generation process, the small size, and the rare occurrence of surface defects. From literature and a polymer products manufacturing use case, we identify design requirements which reflect the aforementioned challenges. Addressing these, we conceptualize design principles and features informed by deep lea…
Rani Anna Government College for Women（拉尼·安娜政府女子学院）
Ceramic tiles have become very popular and are used in the flooring of offices and shopping malls. As testing the quality of tiles manually in a highly polluted environment in the manufacturing industry is a labor-intensive and time consuming process, analysis is carried out on the tile images. This paper discusses an automated system to detect the defects on the surface of ceramic tiles based on dilation, erosion, SMEE and boundary detection techniques.
Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
To ensure flight safety of aircraft structures, it is necessary to have regular maintenance using visual and nondestructive inspection (NDI) methods. In this paper, we propose an automatic image-based aircraft defect detection using Deep Neural Networks (DNNs). To the best of our knowledge, this is the first work for aircraft defect detection using DNNs. We perform a comprehensive evaluation of state-of-the-art feature descriptors and show that the best performance is achieved by vgg-f DNN as feature extractor with a linear SVM classifier. To reduce the processing time, we propose to apply SUR…
Lian Sazeh group（联萨泽集团）；University of Tehran（德黑兰大学）；Deakin University（迪肯大学）；Institute for Intelligent Systems Research and Innovation（智能系统研究与创新研究所）
Defects are unavoidable in casting production owing to the complexity of the casting process. While conventional human-visual inspection of casting products is slow and unproductive in mass productions, an automatic and reliable defect detection not just enhances the quality control process but positively improves productivity. However, casting defect detection is a challenging task due to diversity and variation in defects' appearance. Convolutional neural networks (CNNs) have been widely applied in both image classification and defect detection tasks. Howbeit, CNNs with frequentist inference…
Nowadays, Deep Convolutional Neural Networks (DCNNs) are widely used in fabric defect detection, which come with the cost of expensive training and complex model parameters. With the observation that most fabrics are defect free in practice, a two-step Cascaded Zoom-In Network (CZI-Net) is proposed for patterned fabric defect detection. In the CZI-Net, the Aggregated HOG (A-HOG) and SIFT features are used to instead of simple convolution filters for feature extraction. Moreover, in order to extract more distinctive features, the feature representation layer and full connection layer are includ…
Stanford University（斯坦福大学）；S&P Global（标普全球）；University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校）
Our works experimented DeepLabV3+ with different backbones on a large volume of steel images aiming to automatically detect different types of steel defects. Our methods applied random weighted augmentation to balance different defects types in the training set. And then applied DeeplabV3+ model three different backbones, ResNet, DenseNet and EfficientNet, on segmenting defection regions on the steel images. Based on experiments, we found that applying ResNet101 or EfficientNet as backbones could reach the best IoU scores on the test set, which is around 0.57, comparing with 0.325 for using De…
Airbags are subject to strict quality control in order to ensure passengers safety. The quality of fabric and sewing thread influence the final product and therefore, sewing defects must be early and accurately detected, in order to remove the item from production. Airbag seams assembly can take various forms, using linear and circle primitives, with threads of different colors and length densities, creating lockstitch or double threads chainstitch. The paper presents a framework for the automatic detection of defects occurring during the airbag sewing stage. Types of defects as skipped stitch…
Najafabad Branch, Islamic Azad University（伊斯兰阿扎德大学纳杰法巴德分校）
Texture analysis plays an important role in many image processing applications to describe the image content or objects. On the other hand, visual surface defect detection is a highly research field in the computer vision. Surface defect refers to abnormalities in the texture of the surface. So, in this paper a dual purpose benchmark dataset is proposed for texture image analysis and surface defect detection titled stone texture image (STI dataset). The proposed benchmark dataset consist of 4 different class of stone texture images. The proposed benchmark dataset have some unique properties to…
