Our works experimented DeepLabV3+ with different backbones on a large volume of steel images aiming to automatically detect different types of steel defects. Our methods applied random weighted augmentation to balance different defects types in the training set. And then applied DeeplabV3+ model three different backbones, ResNet, DenseNet and EfficientNet, on segmenting defection regions on the steel images. Based on experiments, we found that applying ResNet101 or EfficientNet as backbones could reach the best IoU scores on the test set, which is around 0.57, comparing with 0.325 for using DenseNet. Also, DeepLabV3+ model with ResNet101 as backbone has the fewest training time.
Automatic defect recognition is one of the research hotspots in steel production, but most of the current methods mainly extract features manually and use machine learning classifiers to recognize defects, which cannot tackle the situation, where there are few data available to train and confine to a certain scene. Therefore, in this paper, a new approach is proposed which consists of part of pretrained VGG16 as a feature extractor and a new CNN neural network as a classifier to recognize the defect of steel strip surface based on the feature maps created by the feature extractor. Our method a…
SPDCN: Strip-based Deformable Convolutional Network for Steel Surface Defect Segmentation
School of Artificial Intelligence, Jiangxi Normal University（江西师范大学人工智能学院）；Jiangxi Provincial Key Laboratory of Intelligent Information Processing and Affective Computing（江西省智能信息处理与情感计算重点实验室）
School of Computer Science, Yangtze University（长江大学计算机科学学院）；School of Electronics & Information, Yangtze University（长江大学电子信息学院）；School of Urban Construction, Yangtze University（长江大学城市建设学院）
Steel pipes are widely used in high-risk and high-pressure scenarios such as oil, chemical, natural gas, shale gas, etc. If there is some defect in steel pipes, it will lead to serious adverse consequences. Applying object detection in the field of deep learning to pipe weld defect detection and identification can effectively improve inspection efficiency and promote the development of industrial automation. Most predecessors used traditional computer vision methods applied to detect defects of steel pipe weld seams. However, traditional computer vision methods rely on prior knowledge and can …
A Steel Surface Defect Detection Method Based on Lightweight Convolution Optimization
Hainan Tropical Ocean University（海南热带海洋学院）；University of Sanya（三亚学院）；Universiti Malaysia Perlis（马来西亚玻璃市大学）；Thalesgroup（泰雷兹集团）；Pukyong National University（釜庆国立大学）
KineticAI Inc.（KineticAI公司）；Indian Institute of Science (IISc)（印度科学学院）；The University of Findlay（芬德利大学）；The University of Toledo（托莱多大学）
Visual steel surface defect detection is an essential step in steel sheet manufacturing. Several machine learning-based automated visual inspection (AVI) methods have been studied in recent years. However, most steel manufacturing industries still use manual visual inspection due to training time and inaccuracies involved with AVI methods. Automatic steel defect detection methods could be useful in less expensive and faster quality control and feedback. But preparing the annotated training data for segmentation and classification could be a costly process. In this work, we propose to use the T…
Quality control is an essential process in manufacturing to make the product defect-free as well as to meet customer needs. The automation of this process is important to maintain high quality along with the high manufacturing throughput. With recent developments in deep learning and computer vision technologies, it has become possible to detect various features from the images with near-human accuracy. However, many of these approaches are data intensive. Training and deployment of such a system on manufacturing floors may become expensive and time-consuming. The need for large amounts of tra…
For steel product manufacturing in indoor factories, steel defect detection is important for quality control. For example, a steel sheet is extremely delicate, and must be accurately inspected. However, to maintain the painted steel parts of the infrastructure around a severe outdoor environment, corrosion detection is critical for predictive maintenance. In this paper, we propose a general-purpose application for steel anomaly detection that consists of the following four components. The first, a learner, is a unit image classification network to determine whether the region of interest or ba…
Integrated Laser Scanning and Image-Based Topology Optimization Techniques for Detection and Quantification of Visible and Subsurface Structural Defects
New Horizon College of Engineering（新视野工程学院）；Govt. College of Engineering（政府工程学院）；University of Mysore（迈索尔大学）
In this paper, we propose a machine vision algorithm for automatically detecting defects in textures belonging to 16 out of 17 wallpaper groups using texture-periodicity and a family of Gabor wavelets. Input defective images are subjected to Gabor wavelet transformation in multi-scales and multi-orientations and a resultant image is obtained in L2 norm. The resultant image is split into several periodic blocks and energy of each block is used as a feature space to automatically identify defective and defect-free blocks using Ward's hierarchical clustering. Experiments on defective fabric image…
This paper addresses the problem of defect segmentation in semiconductor manufacturing. The input of our segmentation is a scanning-electron-microscopy (SEM) image of the candidate defect region. We train a U-net shape network to segment defects using a dataset of clean background images. The samples of the training phase are produced automatically such that no manual labeling is required. To enrich the dataset of clean background samples, we apply defect implant augmentation. To that end, we apply a copy-and-paste of a random image patch in the clean specimen. To improve the robustness of the…
In this paper, we present a Robust Completed Local Binary Pattern (RCLBP) framework for a surface defect detection task. Our approach uses a combination of Non-Local (NL) means filter with wavelet thresholding and Completed Local Binary Pattern (CLBP) to extract robust features which are fed into classifiers for surface defects detection. This paper combines three components: A denoising technique based on Non-Local (NL) means filter with wavelet thresholding is established to denoise the noisy image while preserving the textures and edges. Second, discriminative features are extracted using t…
New Horizon College of Engineering（新视野工程学院）；JSS Research Foundation, SJCE Campus, University of Mysore（迈索尔大学JSS研究基金会SJCE校区）；Government Engineering College（政府工程学院）；University of Mysore（迈索尔大学）
Chi-square histogram distance is one of the distance measures that can be used to find dissimilarity between two histograms. Motivated by the fact that texture discrimination by human vision system is based on second-order statistics, we make use of histogram of gray-level co-occurrence matrix (GLCM) that is based on second-order statistics and propose a new machine vision algorithm for automatic defect detection on patterned textures. Input defective images are split into several periodic blocks and GLCMs are computed after quantizing the gray levels from 0-255 to 0-63 to keep the size of GLC…
New Horizon College of Engineering（新视野工程学院）；Govt. College of Engineering, Chamarajanagar（查马拉贾纳加尔政府工程学院）；University of Mysore（迈索尔大学）
In this paper, we propose a machine vision algorithm for automatically detecting defects in patterned textures with the help of gradient space and its energy. Experiments on real fabric images with defects show that the proposed method can be used for automatic detection of fabric defects in textile industries.
Leather is a natural and durable material created through a process of tanning of hides and skins of animals. The price of the leather is subjective as it is highly sensitive to its quality and surface defects condition. In the literature, there are very few works investigating on the defects detection for leather using automatic image processing techniques. The manual defect inspection process is essential in an leather production industry to control the quality of the finished products. However, it is tedious, as it is labour intensive, time consuming, causes eye fatigue and often prone to h…
Khulna University（库尔纳大学）；Asian University of Bangladesh（孟加拉国亚洲大学）
Quality control is an important issue in the ceramic tile industry. On the other hand maintaining the rate of production with respect to time is also a major issue in ceramic tile manufacturing. Again, price of ceramic tiles also depends on purity of texture, accuracy of color, shape etc. Considering this criteria, an automated defect detection and classification technique has been proposed in this report that can have ensured the better quality of tiles in manufacturing process as well as production rate. Our proposed method plays an important role in ceramic tiles industries to detect the de…
Deep-learning methods have recently started being employed for addressing surface-defect detection problems in industrial quality control. However, with a large amount of data needed for learning, often requiring high-precision labels, many industrial problems cannot be easily solved, or the cost of the solutions would significantly increase due to the annotation requirements. In this work, we relax heavy requirements of fully supervised learning methods and reduce the need for highly detailed annotations. By proposing a deep-learning architecture, we explore the use of annotations of differen…
Segmentation-based, two-stage neural network has shown excellent results in the surface defect detection, enabling the network to learn from a relatively small number of samples. In this work, we introduce end-to-end training of the two-stage network together with several extensions to the training process, which reduce the amount of training time and improve the results on the surface defect detection tasks. To enable end-to-end training we carefully balance the contributions of both the segmentation and the classification loss throughout the learning. We adjust the gradient flow from the cla…
Nuremberg Institute of Technology（纽伦堡技术学院）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；ZAE Bayern（巴伐利亚应用能源研究中心）
Electroluminescence (EL) imaging is a useful modality for the inspection of photovoltaic (PV) modules. EL images provide high spatial resolution, which makes it possible to detect even finest defects on the surface of PV modules. However, the analysis of EL images is typically a manual process that is expensive, time-consuming, and requires expert knowledge of many different types of defects. In this work, we investigate two approaches for automatic detection of such defects in a single image of a PV cell. The approaches differ in their hardware requirements, which are dictated by their respec…
Wislon College of Textiles, North Carolina State University（北卡罗来纳州立大学威尔逊纺织学院）
Automatic defect detection is a challenging task because of the variability in texture and type of fabric defects. An effective defect detection system enables manufacturers to improve the quality of processes and products. Automation across the textile manufacturing systems would reduce fabric wastage and increase profitability by saving cost and resources. There are different contemporary research on automatic defect detection systems using image processing and machine learning techniques. These techniques differ from each other based on the manufacturing processes and defect types. Research…
Oak Ridge National Laboratory（橡树岭国家实验室）；University of Michigan-Ann Arbor（密歇根大学安娜堡分校）；Hope College（霍普学院）；University of Puerto Rico at Mayagüez（波多黎各大学马亚圭斯分校）
Electron microscopy is widely used to explore defects in crystal structures, but human detecting of defects is often time-consuming, error-prone, and unreliable, and is not scalable to large numbers of images or real-time analysis. In this work, we discuss the application of machine learning approaches to find the location and geometry of different defect clusters in irradiated steels. We show that a deep learning based Faster R-CNN analysis system has a performance comparable to human analysis with relatively small training data sets. This study proves the promising ability to apply deep lear…
Stanford University（斯坦福大学）；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）
Segmentation of additive manufacturing (AM) defects in X-ray Computed Tomography (XCT) images is challenging, due to the poor contrast, small sizes and variation in appearance of defects. Automatic segmentation can, however, provide quality control for additive manufacturing. Over recent years, three-dimensional convolutional neural networks (3D CNNs) have performed well in the volumetric segmentation of medical images. In this work, we leverage techniques from the medical imaging domain and propose training a 3D U-Net model to automatically segment defects in XCT images of AM samples. This wo…
School of Electrical and Electronic Engineering, North China Electric Power University（华北电力大学电气与电子工程学院）；School of Computer Science and Technology, North China Electric Power University（华北电力大学计算机科学与技术学院）
Bolts are the most numerous fasteners in transmission lines and are prone to losing their split pins. How to realize the automatic pin-missing defect detection for bolts in transmission lines so as to achieve timely and efficient trouble shooting is a difficult problem and the long-term research target of power systems. In this paper, an automatic detection model called Automatic Visual Shape Clustering Network (AVSCNet) for pin-missing defect is constructed. Firstly, an unsupervised clustering method for the visual shapes of bolts is proposed and applied to construct a defect detection model …
University of Waterloo（滑铁卢大学）；Waterloo Artificial Intelligence Institute（滑铁卢人工智能研究所）
Existing multi-camera solutions for automatic scorekeeping in steel-tip darts are very expensive and thus inaccessible to most players. Motivated to develop a more accessible low-cost solution, we present a new approach to keypoint detection and apply it to predict dart scores from a single image taken from any camera angle. This problem involves detecting multiple keypoints that may be of the same class and positioned in close proximity to one another. The widely adopted framework for regressing keypoints using heatmaps is not well-suited for this task. To address this issue, we instead propo…
Graz University of Technology（格拉茨工业大学）；Medical University of Graz（格拉茨医科大学）
We introduce a fully automatic system for cranial implant design, a common task in cranioplasty operations. The system is currently integrated in Studierfenster (http://studierfenster.tugraz.at/), an online, cloud-based medical image processing platform for medical imaging applications. Enhanced by deep learning algorithms, the system automatically restores the missing part of a skull (i.e., skull shape completion) and generates the desired implant by subtracting the defective skull from the completed skull. The generated implant can be downloaded in the STereoLithography (.stl) format directl…
School of Optical and Electronic Information, Huazhong University of Science and Technology（华中科技大学光学与电子信息学院）
Owing to the lack of defect samples in industrial product quality inspection, trained segmentation model tends to overfit when applied online. To address this problem, we propose a defect sample simulation algorithm based on neural style transfer. The simulation algorithm requires only a small number of defect samples for training, and can efficiently generate simulation samples for next-step segmentation task. In our work, we introduce a masked histogram matching module to maintain color consistency of the generated area and the true defect. To preserve the texture consistency with the surrou…
Real-Time Industrial Defect Detection on Edge Hardware Using Fine-Tuned YOLOv8: A Systematic Benchmark on the NEU Surface Defect Database and MVTec AD with Automotive & Battery Manufacturing Extensions
Lian Sazeh group（联萨泽集团）；University of Tehran（德黑兰大学）；Deakin University（迪肯大学）；Institute for Intelligent Systems Research and Innovation（智能系统研究与创新研究所）
Defects are unavoidable in casting production owing to the complexity of the casting process. While conventional human-visual inspection of casting products is slow and unproductive in mass productions, an automatic and reliable defect detection not just enhances the quality control process but positively improves productivity. However, casting defect detection is a challenging task due to diversity and variation in defects' appearance. Convolutional neural networks (CNNs) have been widely applied in both image classification and defect detection tasks. Howbeit, CNNs with frequentist inference…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Oak Ridge National Laboratory（橡树岭国家实验室）；University of Oxford（牛津大学）；University of Michigan - Ann Arbor（密歇根大学安娜堡分校）
Videos captured using Transmission Electron Microscopy (TEM) can encode details regarding the morphological and temporal evolution of a material by taking snapshots of the microstructure sequentially. However, manual analysis of such video is tedious, error-prone, unreliable, and prohibitively time-consuming if one wishes to analyze a significant fraction of frames for even videos of modest length. In this work, we developed an automated TEM video analysis system for microstructural features based on the advanced object detection model called YOLO and tested the system on an in-situ ion irradi…
Southeast University（东南大学）；Southeast University Shenzhen Research Institute（东南大学深圳研究院）
Nowadays, the rapid development of photovoltaic(PV) power stations requires increasingly reliable maintenance and fault diagnosis of PV modules in the field. Due to the effectiveness, convolutional neural network (CNN) has been widely used in the existing automatic defect detection of PV cells. However, the parameters of these CNN-based models are very large, which require stringent hardware resources and it is difficult to be applied in actual industrial projects. To solve these problems, we propose a novel lightweight high-performance model for automatic defect detection of PV cells in elect…
Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
To ensure flight safety of aircraft structures, it is necessary to have regular maintenance using visual and nondestructive inspection (NDI) methods. In this paper, we propose an automatic image-based aircraft defect detection using Deep Neural Networks (DNNs). To the best of our knowledge, this is the first work for aircraft defect detection using DNNs. We perform a comprehensive evaluation of state-of-the-art feature descriptors and show that the best performance is achieved by vgg-f DNN as feature extractor with a linear SVM classifier. To reduce the processing time, we propose to apply SUR…
Airbags are subject to strict quality control in order to ensure passengers safety. The quality of fabric and sewing thread influence the final product and therefore, sewing defects must be early and accurately detected, in order to remove the item from production. Airbag seams assembly can take various forms, using linear and circle primitives, with threads of different colors and length densities, creating lockstitch or double threads chainstitch. The paper presents a framework for the automatic detection of defects occurring during the airbag sewing stage. Types of defects as skipped stitch…
Leibniz University Hannover（莱布尼茨汉诺威大学）；University of British Columbia（英属哥伦比亚大学）
In industrial manufacturing processes, errors frequently occur at unpredictable times and in unknown manifestations. We tackle the problem of automatic defect detection without requiring any image samples of defective parts. Recent works model the distribution of defect-free image data, using either strong statistical priors or overly simplified data representations. In contrast, our approach handles fine-grained representations incorporating the global and local image context while flexibly estimating the density. To this end, we propose a novel fully convolutional cross-scale normalizing flo…
ISC-Perception: A Hybrid Computer Vision Dataset for Object Detection in Novel Steel Assembly
School of Natural and Built Environment, Queen’s University Belfast（女王大学贝尔法斯特自然与建筑环境学院）；School of Civil & Environmental Engineering, and Construction Management, University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校土木与环境工程学院及建设管理学院）；School of Electronics, Electrical Engineering, and Computer Science, Queen’s University Belfast（女王大学贝尔法斯特电子、电气工程及计算机科学学院）；Department of Civil and Urban Engineering, New York University（纽约大学土木与城市工程系）
AGH University of Science and Technology（AGH科技大学）；University of Applied Sciences Western Switzerland（瑞士西部应用科学大学）；MedApp S.A.（MedApp股份公司）
The goal of this work is to propose a robust, fast, and fully automatic method for personalized cranial defect reconstruction and implant modeling. We propose a two-step deep learning-based method using a modified U-Net architecture to perform the defect reconstruction, and a dedicated iterative procedure to improve the implant geometry, followed by automatic generation of models ready for 3-D printing. We propose a cross-case augmentation based on imperfect image registration combining cases from different datasets. We perform ablation studies regarding different augmentation strategies and c…
Politecnico di Milano（米兰理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Iran University of Science and Technology（伊朗科学技术大学）；Swiss Federal Institute of Technology at Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
A current trend in industries such as semiconductors and foundry is to shift their visual inspection processes to Automatic Visual Inspection (AVI) systems, to reduce their costs, mistakes, and dependency on human experts. This paper proposes a two-staged fault diagnosis framework for AVI systems. In the first stage, a generation model is designed to synthesize new samples based on real samples. The proposed augmentation algorithm extracts objects from the real samples and blends them randomly, to generate new samples and enhance the performance of the image processor. In the second stage, an …
Machine learning approaches for automatic defect detection in photovoltaic systems
Department of Electrical Engineering, Indian Institute of Technology Banaras Hindu University, Varanasi 221005, India（印度班加罗尔 Hindu 大学电子工程系）；Department of Mechanical Engineering, Texas Tech University, Lubbock, Texas 79409, USA（德克萨斯理工大学机械工程系）
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；University of Virginia（弗吉尼亚大学）
Internal properties of a sample can be observed by medical imaging tools, such as ultrasound devices, magnetic resonance imaging (MRI) and optical coherence tomography (OCT) which are based on relying on changes in material density or chemical composition [1-21]. As a preliminary investigation, the feasibility to detect interior defects inferred from the discrepancy in elasticity modulus distribution of a three-dimensional heterogeneous sample using only surface full-field measurements and finite element model updating as an inverse optimization algorithm without any assumption about local hom…
Fasteners play a critical role in securing various parts of machinery. Deformations such as dents, cracks, and scratches on the surface of fasteners are caused by material properties and incorrect handling of equipment during production processes. As a result, quality control is required to ensure safe and reliable operations. The existing defect inspection method relies on manual examination, which consumes a significant amount of time, money, and other resources; also, accuracy cannot be guaranteed due to human error. Automatic defect detection systems have proven impactful over the manual i…
Modern industry requires modern solutions for monitoring the automatic production of goods. Smart monitoring of the functionality of the mechanical parts of technology systems or machines is mandatory for a fully automatic production process. Although Deep Learning has been advancing, allowing for real-time object detection and other tasks, little has been investigated about the effectiveness of specially designed Convolutional Neural Networks for defect detection and industrial object recognition. In the particular study, we employed six publically available industrial-related datasets contai…
