电力公司越来越多地依赖无人机图像进行事后和常规检查，但训练准确的缺陷类型分类器仍然困难，因为缺陷示例很少且检查数据集通常有限或专有。我们通过使用现成的多模态大语言模型（MLLM）作为无训练图像生成器来解决数据稀缺的问题，该模型可以基于视觉参考和文本提示合成缺陷图像。我们的流程通过双参考条件增加多样性，通过轻量级人工验证和提示优化提高标签保真度，并使用基于嵌入的筛选规则过滤所生成的合成池，该规则基于从真实训练集计算的类别质心的距离。我们在陶瓷绝缘子缺陷类型分类（壳 vs. 胶层）上评估，使用一个公开数据集，在现实的低训练数据情况下（104张真实训练图像；152张验证；308张测试）。将10%的真实训练集与基于嵌入选择的合成图像增强后，测试F1分数（精度和召回率的调和平均）从0.615提高到0.739（20%的提升），对应于估计的4-5倍的数据效率提升，且在更强的骨干模型和冻结特征线性探测基线中收益仍然存在。这些结果表明，当收集额外真实缺陷较慢或不可行时，提供了一条实用且低门槛的改进缺陷识别的方法。
英文摘要：
Utility companies increasingly rely on drone imagery for post-event and routine inspection, but training accurate defect-type classifiers remains difficult because defect examples are rare and inspection datasets are often limited or proprietary. We address this data-scarcity setting by using an off-the-shelf multimodal large language model (MLLM) as a training-free image generator to synthesize defect images from visual references and text prompts. Our pipeline increases diversity via dual-reference conditioning, improves label fidelity with lightweight human verification and prompt refinement, and filters the resulting synthetic pool using an embedding-based selection rule based on distances to class centroids computed from the real training split. We evaluate on ceramic insulator defect-type classification (shell vs. glaze) using a public dataset with a realistic low training-data regime (104 real training images; 152 validation; 308 test). Augmenting the 10% real training set with embedding-selected synthetic images improves test F1 score (harmonic mean of precision and recall) from 0.615 to 0.739 (20% relative), corresponding to an estimated 4--5x data-efficiency gain, and the gains persist with stronger backbone models and frozen-feature linear-probe baselines. These results suggest a practical, low-barrier path for improving defect recognition when collecting additional real defects is slow or infeasible.
Nanyang Technological University（南洋理工大学）；Delta Research Center（台达研究院）
Automated defect inspection is critical for effective and efficient maintenance, repair, and operations in advanced manufacturing. On the other hand, automated defect inspection is often constrained by the lack of defect samples, especially when we adopt deep neural networks for this task. This paper presents Defect-GAN, an automated defect synthesis network that generates realistic and diverse defect samples for training accurate and robust defect inspection networks. Defect-GAN learns through defacement and restoration processes, where the defacement generates defects on normal surface image…
Tsinghua University（清华大学）；University of Cincinnati（辛辛那提大学）
Visual surface inspection is a challenging task owing to the highly diverse appearance of target surfaces and defective regions. Previous attempts heavily rely on vast quantities of training examples with manual annotation. However, in some practical cases, it is difficult to obtain a large number of samples for inspection. To combat it, we propose a hierarchical texture-perceiving generative adversarial network (HTP-GAN) that is learned from the one-shot normal image in an unsupervised scheme. Specifically, the HTP-GAN contains a pyramid of convolutional GANs that can capture the global struc…
The quality of industrial components is critical to the production of special equipment such as robots. Defect inspection of these components is an efficient way to ensure quality. In this paper, we propose a hybrid network, SSD-Faster Net, for industrial defect inspection of rails, insulators, commutators etc. SSD-Faster Net is a two-stage network, including SSD for quickly locating defective blocks, and an improved Faster R-CNN for defect segmentation. For the former, we propose a novel slice localization mechanism to help SSD scan quickly. The second stage is based on improved Faster R-CNN,…
MoE Key Lab of Artificial Intelligence（教育部人工智能重点实验室）；Shanghai Jiao Tong University（上海交通大学）
The performances of defect inspection have been severely hindered by insufficient defect images in industries, which can be alleviated by generating more samples as data augmentation. We propose the first defect image generation method in the challenging few-shot cases. Given just a handful of defect images and relatively more defect-free ones, our goal is to augment the dataset with new defect images. Our method consists of two training stages. First, we train a data-efficient StyleGAN2 on defect-free images as the backbone. Second, we attach defect-aware residual blocks to the backbone, whic…
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；National Technical University of Athens（雅典国立技术大学）；Philips Consumer Lifestyle BV（飞利浦消费者生活方式公司）；Qlector d.o.o.（Qlector公司）
Quality control is a crucial activity performed by manufacturing companies to ensure their products conform to the requirements and specifications. The introduction of artificial intelligence models enables to automate the visual quality inspection, speeding up the inspection process and ensuring all products are evaluated under the same criteria. In this research, we compare supervised and unsupervised defect detection techniques and explore data augmentation techniques to mitigate the data imbalance in the context of automated visual inspection. Furthermore, we use Generative Adversarial Net…
Quality inspection applications in industry are required to move towards a zero-defect manufacturing scenario, withnon-destructive inspection and traceability of 100 % of produced parts. Developing robust fault detection and classification modelsfrom the start-up of the lines is challenging due to the difficulty in getting enough representative samples of the faulty patternsand the need to manually label them. This work presents a methodology to develop a robust inspection system, targeting thesepeculiarities, in the context of solar cell manufacturing. The methodology is divided into two phas…
From Prediction to Diagnosis: Reasoning-Aware AI for Photovoltaic Defect Inspection
Illinois Institute of Technology（伊利诺伊理工学院）；Argonne National Laboratory（阿贡国家实验室）；Commonwealth Edison（联邦爱迪生公司）；Drexel University（德雷塞尔大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
The vast network of bridges in the United States raises a high requirement for maintenance and rehabilitation. The massive cost of manual visual inspection to assess bridge conditions is a burden to some extent. Advanced robots have been leveraged to automate inspection data collection. Automating the segmentations of multiclass elements and surface defects on the elements in the large volume of inspection image data would facilitate an efficient and effective assessment of the bridge condition. Training separate single-task networks for element parsing (i.e., semantic segmentation of multicla…
Prime University（Prime大学）；Barisal University（巴里萨尔大学）；Independent University, Bangladesh（孟加拉国独立大学）
Quality control at each stage of production in textile industry has become a key factor to retaining the existence in the highly competitive global market. Problems of manual fabric defect inspection are lack of accuracy and high time consumption, where early and accurate fabric defect detection is a significant phase of quality control. Computer vision based, i.e. automated fabric defect inspection systems are thought by many researchers of different countries to be very useful to resolve these problems. There are two major challenges to be resolved to attain a successful automated fabric def…
Defect Spectrum: A Granular Look of Large-Scale Defect Datasets with Rich Semantics
Hong Kong University of Science and Technology, Guangzhou（香港科技大学（广州））；HKUST(GZ) - SmartMore Joint Lab（香港科技大学（广州）- 赛摩智能联合实验室）；SmartMore. Corp（赛摩智能公司）；Hong Kong University of Science and Technology（香港科技大学）
UniPCB: A Generation-Assisted Vision-Based Measurement Framework for PCB Defect Inspection
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Beijing Institute of Technology（北京理工大学）；National Astronomical Observatories of China（中国科学院国家天文台）；Beijing Institute of Technology Chongqing Innovation Center（北京理工大学重庆创新中心）
The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is the world's largest single-dish radio telescope. Its large reflecting surface achieves unprecedented sensitivity but is prone to damage, such as dents and holes, caused by naturally-occurring falling objects. Hence, the timely and accurate detection of surface defects is crucial for FAST's stable operation. Conventional manual inspection involves human inspectors climbing up and examining the large surface visually, a time-consuming and potentially unreliable process. To accelerate the inspection process and increase its accur…
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（达尔文人工智能公司）；Moog Inc.（穆格公司）
In electronics manufacturing, solder joint defects are a common problem affecting a variety of printed circuit board components. To identify and correct solder joint defects, the solder joints on a circuit board are typically inspected manually by trained human inspectors, which is a very time-consuming and error-prone process. To improve both inspection efficiency and accuracy, in this work we describe an explainable deep learning-based visual quality inspection system tailored for visual inspection of solder joints in electronics manufacturing environments. At the core of this system is an e…
Towards Improved Semiconductor Defect Inspection for high-NA EUVL based on SEMI-SuperYOLO-NAS
Faculty of Science, KU Leuven（KU Leuven 理学院）；Interuniversity Microelectronics Centre (imec)（校际微电子中心（imec））；Faculty of Engineering, Ghent University（根特大学 工程学院）；SCREEN SPE Germany GmbH（SCREEN SPE 德国有限公司）
Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Recent advances in the industrial inspection of textured surfaces-in the form of visual inspection-have made such inspections possible for efficient, flexible manufacturing systems. We propose an unsupervised feature memory rearrangement network (FMR-Net) to accurately detect various textural defects simultaneously. Consistent with mainstream methods, we adopt the idea of background reconstruction; however, we innovatively utilize artificial synthetic defects to enable the model to recognize anomalies, while traditional wisdom relies only on defect-free samples. First, we employ an encoding mo…
Establishing Reality-Virtuality Interconnections in Urban Digital Twins for Superior Intelligent Road Inspection and Simulation
College of Electronics & Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University, Shanghai 201804, China（电子与信息工程学院，上海智能自主系统研究院，智能自主系统国家重点实验室，智能自主系统前沿科学中心，同济大学，上海）
Czech Technical University in Prague（捷克技术大学）；IBM Research Zürich（IBM苏黎世研究院）
Labeling images for visual segmentation is a time-consuming task which can be costly, particularly in application domains where labels have to be provided by specialized expert annotators, such as civil engineering. In this paper, we propose to use attribution methods to harness the valuable interactions between expert annotators and the data to be annotated in the case of defect segmentation for visual inspection of civil infrastructures. Concretely, a classifier is trained to detect defects and coupled with an attribution-based method and adversarial climbing to generate and refine segmentat…
Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Costa Rica Institute of Technology（哥斯达黎加理工学院）；Gestalt Automation GmbH（Gestalt Automation有限公司）；Technical University Berlin（柏林工业大学）
Facility for Antiproton and Ion Research in Europe（欧洲反质子与离子研究装置）
Optical inspection of 1191 silicon micro-strip sensors was performed using a custom made optical inspection setup, employing a machine-learning based approach for the defect analysis and subsequent quality assurance. Furthermore, metrological control of the sensor's surface was performed. In this manuscript, we present the analysis of various sensor surface defects. Among these are implant breaks, p-stop breaks, aluminium strip opens, aluminium strip shorts, surface scratches, double metallization layer defects, passivation layer defects, bias resistor defects as well as dust particle identifi…
Deep learning-based object detection is a powerful approach for detecting faulty insulators in power lines. This involves training an object detection model from scratch, or fine tuning a model that is pre-trained on benchmark computer vision datasets. This approach works well with a large number of insulator images, but can result in unreliable models in the low data regime. The current literature mainly focuses on detecting the presence or absence of insulator caps, which is a relatively easy detection task, and does not consider detection of finer faults such as flashed and broken disks. In…
In industry, Deep Neural Networks have shown high defect detection rates surpassing other more traditional manual feature engineering based proposals. This has been achieved mainly through supervised training where a great amount of data is required in order to learn good classification models. However, such amount of data is sometimes hard to obtain in industrial scenarios, as few defective pieces are produced normally. In addition, certain kinds of defects are very rare and usually just appear from time to time, which makes the generation of a proper dataset for training a classification mod…
Defect Segmentation in OCT scans of ceramic parts for non-destructive inspection using deep learning
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València, Valencia, Spain Department of Electrical；Photonics Engineering, Technical University of Denmark, Kongens Lyngby, Denmark NORBLIS ApS, Virum, Denmark Lithoz GmbH, Vienna, Austria airCode UG, Duisburg, Germany
SenseTime Research（商汤科技研究院）；JD.com Inc Qing Yuan Research Institute（京东集团清源研究院）；Shanghai Jiao Tong University（上海交通大学）
Visual anomaly detection plays a crucial role in not only manufacturing inspection to find defects of products during manufacturing processes, but also maintenance inspection to keep equipment in optimum working condition particularly outdoors. Due to the scarcity of the defective samples, unsupervised anomaly detection has attracted great attention in recent years. However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying cam…
The usage of Unmanned Aerial Vehicles (UAVs) in the context of structural health inspection is recently gaining tremendous popularity. Camera mounted UAVs enable the fast acquisition of a large number of images often used for mapping, 3D model reconstruction, and as an assisting tool for inspectors. Due to the number of images captured during large scale UAV surveys, a manual image-based inspection analysis of entire assets cannot be efficiently performed by qualified engineers. Additionally, comparing defects to past inspections requires the retrieval of relevant images which is often impract…
The Chinese University of Hong Kong（香港中文大学）；Minjiang University（闽江学院）
Automatic crack detection and segmentation play a significant role in the whole system of unmanned aerial vehicle inspections. In this paper, we have implemented a deep learning framework for crack detection based on classical network architectures including Alexnet, VGG, and Resnet. Moreover, inspired by the feature pyramid network architecture, a hierarchical convolutional neural network (CNN) deep learning framework which is efficient in crack segmentation is also proposed, and its performance of it is compared with other state-of-the-art network architecture. We have summarized the existin…
We are proceeding towards the age of automation and robotic integration of our production lines [5]. Effective quality-control systems have to be put in place to maintain the quality of manufactured components. Among different quality-control systems, vision-based inspection systems have gained considerable amount of popularity [8] due to developments in computing power and image processing techniques. In this paper, we present a vision-based inspection system (VBI) as a quality-control system, which not only detects the presence of defects, such as in conventional VBIs, but also leverage deve…
An Incremental Unified Framework for Small Defect Inspection
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Chinese University of Hong Kong（香港中文大学）；SmartMore Corporation（思谋科技公司）；Hong Kong Industrial Artificial Intelligence & Robotics Centre（香港工业人工智能及机器人中心）；Hong Kong Productivity Council（香港生产力促进局）
A Dataset and Baseline for Deep Learning-Based Visual Quality Inspection in Remanufacturing
Institute for Machine Tools and Industrial Management (iwb), Technical University of Munich（慕尼黑工业大学机床与工业管理研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎高科矿业学院，巴黎文理研究大学，数学形态学中心）；Fraunhofer Institute for Casting, Composite and Processing Technology IGCV（弗劳恩霍夫铸造、复合材料与加工技术研究所）
The presence of any type of defect on the glass screen of smart devices has a great impact on their quality. We present a robust semi-supervised learning framework for intelligent micro-scaled localization and classification of defects on a 16K pixel image of smartphone glass. Our model features the efficient recognition and labeling of three types of defects: scratches, light leakage due to cracks, and pits. Our method also differentiates between the defects and light reflections due to dust particles and sensor regions, which are classified as non-defect areas. We use a partially labeled dat…
Attention Modules Improve Image-Level Anomaly Detection for Industrial Inspection: A DifferNet Case Study
Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco（佩南布科联邦大学信息学中心 Voxar 实验室）；Visual Computing Lab, DC, Universidade Federal Rural de Pernambuco（佩南布科联邦农村大学 DC 视觉计算实验室）；Chemnitz University of Technology（开姆尼茨工业大学）
Detecting and Classifying Defective Products in Images Using YOLO
Northeastern University（东北大学）；Florida International University（佛罗里达国际大学）；University of California San Diego（加州大学圣地亚哥分校）；Tulane University（杜兰大学）；Kanazawa University（金泽大学）；Northern Arizona University（北亚利桑那大学）
Unicom (Shanghai) Industry Internet Co., Ltd.（联通（上海）产业互联网有限公司）
Currently, most deep learning methods cannot solve the problem of scarcity of industrial product defect samples and significant differences in characteristics. This paper proposes an unsupervised defect detection algorithm based on a reconstruction network, which is realized using only a large number of easily obtained defect-free sample data. The network includes two parts: image reconstruction and surface defect area detection. The reconstruction network is designed through a fully convolutional autoencoder with a lightweight structure. Only a small number of normal samples are used for trai…
Center for Energy Science；Technology, Skolkovo Institute of Science
Power transmission networks physically connect the power generators to the electric consumers. Such systems extend over hundreds of kilometers. There are many components in the transmission infrastructure that require a proper inspection to guarantee flawless performance and reliable delivery, which, if done manually, can be very costly and time consuming. One essential component is the insulator. Its failure can cause an interruption of the entire transmission line or a widespread power failure. Automated fault detection could significantly decrease inspection time and related costs. Recently…
