Segmentation-based, two-stage neural network has shown excellent results in the surface defect detection, enabling the network to learn from a relatively small number of samples. In this work, we introduce end-to-end training of the two-stage network together with several extensions to the training process, which reduce the amount of training time and improve the results on the surface defect detection tasks. To enable end-to-end training we carefully balance the contributions of both the segmentation and the classification loss throughout the learning. We adjust the gradient flow from the classification into the segmentation network in order to prevent the unstable features from corrupting the learning. As an additional extension to the learning, we propose frequency-of-use sampling scheme of negative samples to address the issue of over- and under-sampling of images during the training, while we employ the distance transform algorithm on the region-based segmentation masks as weights for positive pixels, giving greater importance to areas with higher probability of presence of defect without requiring a detailed annotation. We demonstrate the performance of the end-to-end training scheme and the proposed extensions on three defect detection datasets - DAGM, KolektorSDD and Severstal Steel defect dataset - where we show state-of-the-art results. On the DAGM and the KolektorSDD we demonstrate 100\% detection rate, therefore completely solving the datasets. Additional ablation study performed on all three datasets quantitatively demonstrates the contribution to the overall result improvements for each of the proposed extensions.
CINFormer: Transformer network with multi-stage CNN feature injection for surface defect segmentation
Zhengzhou University（郑州大学）；Engineering Research Center of Intelligent Swarm Systems, Ministry of Education（教育部智能集群系统工程研究中心）；National Supercomputing Center in Zhengzhou（国家超级计算郑州中心）；Tianjin University（天津大学）；The University of Sydney（悉尼大学）；Sydney AI Centre（悉尼人工智能中心）
Southeast University（东南大学）；Southeast University Shenzhen Research Institute（东南大学深圳研究院）
Nowadays, the rapid development of photovoltaic(PV) power stations requires increasingly reliable maintenance and fault diagnosis of PV modules in the field. Due to the effectiveness, convolutional neural network (CNN) has been widely used in the existing automatic defect detection of PV cells. However, the parameters of these CNN-based models are very large, which require stringent hardware resources and it is difficult to be applied in actual industrial projects. To solve these problems, we propose a novel lightweight high-performance model for automatic defect detection of PV cells in elect…
To coupe with the difficulties in the process of inspection and classification of defects in Printed Circuit Board (PCB), other researchers have proposed many methods. However, few of them published their dataset before, which hindered the introduction and comparison of new methods. In this paper, we published a synthesized PCB dataset containing 1386 images with 6 kinds of defects for the use of detection, classification and registration tasks. Besides, we proposed a reference based method to inspect and trained an end-to-end convolutional neural network to classify the defects. Unlike conven…
Goethe University（歌德大学）；Egnatia Odos A. E.（埃格纳蒂亚公路公司）
Recognition of defects in concrete infrastructure, especially in bridges, is a costly and time consuming crucial first step in the assessment of the structural integrity. Large variation in appearance of the concrete material, changing illumination and weather conditions, a variety of possible surface markings as well as the possibility for different types of defects to overlap, make it a challenging real-world task. In this work we introduce the novel COncrete DEfect BRidge IMage dataset (CODEBRIM) for multi-target classification of five commonly appearing concrete defects. We investigate and…
Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics
Hong Kong University of Science and Technology, Guangzhou（香港科技大学（广州））；HKUST(GZ) - SmartMore Joint Lab（香港科技大学（广州）- 赛摩智能联合实验室）；SmartMore. Corp（赛摩智能公司）；Hong Kong University of Science and Technology（香港科技大学）
Fasteners play a critical role in securing various parts of machinery. Deformations such as dents, cracks, and scratches on the surface of fasteners are caused by material properties and incorrect handling of equipment during production processes. As a result, quality control is required to ensure safe and reliable operations. The existing defect inspection method relies on manual examination, which consumes a significant amount of time, money, and other resources; also, accuracy cannot be guaranteed due to human error. Automatic defect detection systems have proven impactful over the manual i…
Investigating Shift Equivalence of Convolutional Neural Networks in Industrial Defect Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Binzhou Institute of Technology（滨州理工学院）；Institute of Automation, Gansu Academy of Sciences（甘肃省科学院自动化研究所）
Technical University of Kaiserslautern（凯泽斯劳滕工业大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Surface inspection systems are an important application domain for computer vision, as they are used for defect detection and classification in the manufacturing industry. Existing systems use hand-crafted features which require extensive domain knowledge to create. Even though Convolutional neural networks (CNNs) have proven successful in many large-scale challenges, industrial inspection systems have yet barely realized their potential due to two significant challenges: real-time processing speed requirements and specialized narrow domain-specific datasets which are sometimes limited in size…
Airbags are subject to strict quality control in order to ensure passengers safety. The quality of fabric and sewing thread influence the final product and therefore, sewing defects must be early and accurately detected, in order to remove the item from production. Airbag seams assembly can take various forms, using linear and circle primitives, with threads of different colors and length densities, creating lockstitch or double threads chainstitch. The paper presents a framework for the automatic detection of defects occurring during the airbag sewing stage. Types of defects as skipped stitch…
Detecting and evaluating surface coating defects is important for marine vessel maintenance. Currently, the assessment is carried out manually by qualified inspectors using international standards and their own experience. Automating the processes is highly challenging because of the high level of variation in vessel type, paint surface, coatings, lighting condition, weather condition, paint colors, areas of the vessel, and time in service. We present a novel deep learning-based pipeline to detect and evaluate the percentage of corrosion, fouling, and delamination on the vessel surface from no…
Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
To ensure flight safety of aircraft structures, it is necessary to have regular maintenance using visual and nondestructive inspection (NDI) methods. In this paper, we propose an automatic image-based aircraft defect detection using Deep Neural Networks (DNNs). To the best of our knowledge, this is the first work for aircraft defect detection using DNNs. We perform a comprehensive evaluation of state-of-the-art feature descriptors and show that the best performance is achieved by vgg-f DNN as feature extractor with a linear SVM classifier. To reduce the processing time, we propose to apply SUR…
National Taipei University of Nursing and Health Sciences（国立台北护理健康大学）；Feng Chia University（逢甲大学）；Xiamen University（厦门大学）；Ton Duc Thang University（孙德胜大学）
Genuine leather, such as the hides of cows, crocodiles, lizards and goats usually contain natural and artificial defects, like holes, fly bites, tick marks, veining, cuts, wrinkles and others. A traditional solution to identify the defects is by manual defect inspection, which involves skilled experts. It is time consuming and may incur a high error rate and results in low productivity. This paper presents a series of automatic image processing processes to perform the classification of leather defects by adopting deep learning approaches. Particularly, the leather images are first partitioned…
ITP Aero（ITP航空）；Univ. Carlos III de Madrid（马德里卡洛斯三世大学）
Industrial X-ray analysis is common in aerospace, automotive or nuclear industries where structural integrity of some parts needs to be guaranteed. However, the interpretation of radiographic images is sometimes difficult and may lead to two experts disagree on defect classification. The Automated Defect Recognition (ADR) system presented herein will reduce the analysis time and will also help reducing the subjective interpretation of the defects while increasing the reliability of the human inspector. Our Convolutional Neural Network (CNN) model achieves 94.2\% accuracy (mAP@IoU=50\%), which …
Quality control is a fundamental component of many manufacturing processes, especially those involving casting or welding. However, manual quality control procedures are often time-consuming and error-prone. In order to meet the growing demand for high-quality products, the use of intelligent visual inspection systems is becoming essential in production lines. Recently, Convolutional Neural Networks (CNNs) have shown outstanding performance in both image classification and localization tasks. In this article, a system is proposed for the identification of casting defects in X-ray images, based…
Feng Chia University（逢甲大学）；National Taipei University of Nursing and Health Sciences（台北护理健康大学）；Xiamen University（厦门大学）；Xiamen University Malaysia（厦门大学马来西亚分校）
Leather is a type of natural, durable, flexible, soft, supple and pliable material with smooth texture. It is commonly used as a raw material to manufacture luxury consumer goods for high-end customers. To ensure good quality control on the leather products, one of the critical processes is the visual inspection step to spot the random defects on the leather surfaces and it is usually conducted by experienced experts. This paper presents an automatic mechanism to perform the leather defect classification. In particular, we focus on detecting tick-bite defects on a specific type of calf leather…
Nondestructive testing (NDT) is widely applied to defect identification of turbine components during manufacturing and operation. Operational efficiency is key for gas turbine OEM (Original Equipment Manufacturers). Automating the inspection process as much as possible, while minimizing the uncertainties involved, is thus crucial. We propose a model based on RetinaNet to identify drilling defects in X-ray images of turbine blades. The application is challenging due to the large image resolutions in which defects are very small and hardly captured by the commonly used anchor sizes, and also due…
Coronary angiography is considered to be a safe tool for the evaluation of coronary artery disease and perform in approximately 12 million patients each year worldwide. [1] In most cases, angiograms are manually analyzed by a cardiologist. Actually, there are no clinical practice algorithms which could improve and automate this work. Neural networks show high efficiency in tasks of image analysis and they can be used for the analysis of angiograms and facilitate diagnostics. We have developed an algorithm based on Convolutional Neural Network and Neural Network U-Net [2] for vessels segmentati…
A Two-Stage Detection-Tracking Framework for Stable Apple Quality Inspection in Dense Conveyor-Belt Environments
Interdisciplinary Program in Artificial Intelligence, Seoul National University（人工智能交叉学科项目，首尔国立大学）；Department of Biological Environmental Science, College of Life Science and Biotechnology, Dongguk University（生命科学与生物技术学院生物环境科学系，东国大学）；Department of Food Science and Biotechnology, Dongguk University（食品科学与生物技术系，东国大学）
Wislon College of Textiles, North Carolina State University（北卡罗来纳州立大学威尔逊纺织学院）
Automatic defect detection is a challenging task because of the variability in texture and type of fabric defects. An effective defect detection system enables manufacturers to improve the quality of processes and products. Automation across the textile manufacturing systems would reduce fabric wastage and increase profitability by saving cost and resources. There are different contemporary research on automatic defect detection systems using image processing and machine learning techniques. These techniques differ from each other based on the manufacturing processes and defect types. Research…
Most of traditional single image deblurring methods before deep learning adopt a coarse-to-fine scheme that estimates a sharp image at a coarse scale and progressively refines it at finer scales. While this scheme has also been adopted to several deep learning-based approaches, recently a number of single-scale approaches have been introduced showing superior performance to previous coarse-to-fine approaches both in quality and computation time. In this paper, we revisit the coarse-to-fine scheme, and analyze defects of previous coarse-to-fine approaches that degrade their performance. Based o…
School of Optical and Electronic Information, Huazhong University of Science and Technology（华中科技大学光学与电子信息学院）
Owing to the lack of defect samples in industrial product quality inspection, trained segmentation model tends to overfit when applied online. To address this problem, we propose a defect sample simulation algorithm based on neural style transfer. The simulation algorithm requires only a small number of defect samples for training, and can efficiently generate simulation samples for next-step segmentation task. In our work, we introduce a masked histogram matching module to maintain color consistency of the generated area and the true defect. To preserve the texture consistency with the surrou…
Aalborg University（奥尔堡大学）；Computer Vision Center（计算机视觉中心）；Autonomous University of Barcelona（巴塞罗那自治大学）；Universitat de Barcelona（巴塞罗那大学）
The sewerage infrastructure is one of the most important and expensive infrastructures in modern society. In order to efficiently manage the sewerage infrastructure, automated sewer inspection has to be utilized. However, while sewer defect classification has been investigated for decades, little attention has been given to classifying sewer pipe properties such as water level, pipe material, and pipe shape, which are needed to evaluate the level of sewer pipe deterioration. In this work we classify sewer pipe defects and properties concurrently and present a novel decoder-focused multi-task c…
As solar capacity installed worldwide continues to grow, there is an increasing awareness that advanced inspection systems are becoming of utmost importance to schedule smart interventions and minimize downtime likelihood. In this work we propose a novel automatic multi-stage model to detect panel defects on aerial images captured by unmanned aerial vehicle by using the YOLOv3 network and Computer Vision techniques. The model combines detections of panels and defects to refine its accuracy and exhibits an average inference time per image of 0.98 s. The main novelties are represented by its ver…
In this work, we introduce a method and present an improved neural work to perform product re-identification, which is an essential core function of a fully automated product defect detection system. Our method is based on feature distance. It is the combination of feature extraction neural networks, such as VGG16, AlexNet, with an image search engine - Vearch. The dataset that we used to develop product re-identification systems is a water-bottle dataset that consists of 400 images of 18 types of water bottles. This is a small dataset, which was the biggest challenge of our work. However, the…
Decision Fusion Network with Perception Fine-tuning for Defect Classification
Zhengzhou University（郑州大学）；Engineering Research Center of Intelligent Swarm Systems, Ministry of Education（教育部智能集群系统工程研究中心）；National Supercomputing Center in Zhengzhou（国家超级计算郑州中心）；Beihang University（北京航空航天大学）
Shandong University（山东大学）；University of Leeds（利兹大学）
This research proposes a Ground Penetrating Radar (GPR) data processing method for non-destructive detection of tunnel lining internal defects, called defect segmentation. To perform this critical step of automatic tunnel lining detection, the method uses a CNN called Segnet combined with the Lovász softmax loss function to map the internal defect structure with GPR synthetic data, which improves the accuracy, automation and efficiency of defects detection. The novel method we present overcomes several difficulties of traditional GPR data interpretation as demonstrated by an evaluation on both…
Similar and indeterminate defect detection of solar cell surface with heterogeneous texture and complex background is a challenge of solar cell manufacturing. The traditional manufacturing process relies on human eye detection which requires a large number of workers without a stable and good detection effect. In order to solve the problem, a visual defect detection method based on multi-spectral deep convolutional neural network (CNN) is designed in this paper. Firstly, a selected CNN model is established. By adjusting the depth and width of the model, the influence of model depth and kernel …
School of Computer Science, Yangtze University（长江大学计算机科学学院）；School of Electronics & Information, Yangtze University（长江大学电子信息学院）；School of Urban Construction, Yangtze University（长江大学城市建设学院）
Steel pipes are widely used in high-risk and high-pressure scenarios such as oil, chemical, natural gas, shale gas, etc. If there is some defect in steel pipes, it will lead to serious adverse consequences. Applying object detection in the field of deep learning to pipe weld defect detection and identification can effectively improve inspection efficiency and promote the development of industrial automation. Most predecessors used traditional computer vision methods applied to detect defects of steel pipe weld seams. However, traditional computer vision methods rely on prior knowledge and can …
The quality of industrial components is critical to the production of special equipment such as robots. Defect inspection of these components is an efficient way to ensure quality. In this paper, we propose a hybrid network, SSD-Faster Net, for industrial defect inspection of rails, insulators, commutators etc. SSD-Faster Net is a two-stage network, including SSD for quickly locating defective blocks, and an improved Faster R-CNN for defect segmentation. For the former, we propose a novel slice localization mechanism to help SSD scan quickly. The second stage is based on improved Faster R-CNN,…
Vali-e-Asr University of Rafsanjan（拉夫桑詹瓦利阿斯尔大学）；Rafsanjan University of Medical Sciences（拉夫桑詹医科大学）
Pistachio nut is mainly consumed as raw, salted or roasted because of its high nutritional properties and favorable taste. Pistachio nuts with shell and kernel defects, besides not being acceptable for a consumer, are also prone to insects damage, mold decay, and aflatoxin contamination. In this research, a deep learning-based imaging algorithm was developed to improve the sorting of nuts with shell and kernel defects that indicate the risk of aflatoxin contamination, such as dark stains, oily stains, adhering hull, fungal decay and Aspergillus molds. This paper presents an unsupervised learni…
Nanyang Technological University（南洋理工大学）；Delta Research Center（台达研究院）
Automated defect inspection is critical for effective and efficient maintenance, repair, and operations in advanced manufacturing. On the other hand, automated defect inspection is often constrained by the lack of defect samples, especially when we adopt deep neural networks for this task. This paper presents Defect-GAN, an automated defect synthesis network that generates realistic and diverse defect samples for training accurate and robust defect inspection networks. Defect-GAN learns through defacement and restoration processes, where the defacement generates defects on normal surface image…
GDDS: A Single Domain Generalized Defect Detection Frame of Open World Scenario using Gather and Distribute Domain-shift Suppression Network
College of Artificial Intelligence and Data Science（人工智能与数据科学学院）；College of Electronic Science（电子科学学院）；Tianjin Advanced Technology Research Institute（天津先进技术研究院）
Politecnico di Milano（米兰理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Iran University of Science and Technology（伊朗科学技术大学）；Swiss Federal Institute of Technology at Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
A current trend in industries such as semiconductors and foundry is to shift their visual inspection processes to Automatic Visual Inspection (AVI) systems, to reduce their costs, mistakes, and dependency on human experts. This paper proposes a two-staged fault diagnosis framework for AVI systems. In the first stage, a generation model is designed to synthesize new samples based on real samples. The proposed augmentation algorithm extracts objects from the real samples and blends them randomly, to generate new samples and enhance the performance of the image processor. In the second stage, an …
An Integrated Hardware-Software Design for Low-Data Spatial Defect Detection in Robotic Visual Inspection with Hybrid Optoelectronic Neural Networks
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；China Belt and Road Joint Lab on Measurement and Control（中国一带一路测量与控制联合实验室）；School of Electric and Electrical Engineering, Chongqing University of Technology（重庆理工大学电气与电子工程学院）；Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）
Defect detection and classification technology has changed from traditional artificial visual inspection to current intelligent automated inspection, but most of the current defect detection methods are training related detection models based on a data-driven approach, taking into account the difficulty of collecting some sample data in the industrial field. We apply zero-shot learning technology to the industrial field. Aiming at the problem of the existing "Latent Feature Guide Attribute Attention" (LFGAA) zero-shot image classification network, the output latent attributes and artificially …
NOVA LINCS（NOVA计算机与系统实验室）；Universidade da Beira Interior（贝拉内陆大学）
The dominant approach for surface defect detection is the use of hand-crafted feature-based methods. However, this falls short when conditions vary that affect extracted images. So, in this paper, we sought to determine how well several state-of-the-art Convolutional Neural Networks perform in the task of surface defect detection. Moreover, we propose two methods: CNN-Fusion, that fuses the prediction of all the networks into a final one, and Auto-Classifier, which is a novel proposal that improves a Convolutional Neural Network by modifying its classification component using AutoML. We carrie…
Prime University（Prime大学）；Barisal University（巴里萨尔大学）；Independent University, Bangladesh（孟加拉国独立大学）
Quality control at each stage of production in textile industry has become a key factor to retaining the existence in the highly competitive global market. Problems of manual fabric defect inspection are lack of accuracy and high time consumption, where early and accurate fabric defect detection is a significant phase of quality control. Computer vision based, i.e. automated fabric defect inspection systems are thought by many researchers of different countries to be very useful to resolve these problems. There are two major challenges to be resolved to attain a successful automated fabric def…
