The design of an automatic visual inspection system is usually performed in two stages. While the first stage consists in selecting the most suitable hardware setup for highlighting most effectively the defects on the surface to be inspected, the second stage concerns the development of algorithmic solutions to exploit the potentials offered by the collected data.
In this paper, first, we present a novel illumination setup embedding four illumination configurations to resemble diffused, dark-field, and front lighting techniques. Second, we analyze the contributions brought by deploying the proposed setup in training phase only - mimicking the scenario in which an already developed visual inspection system cannot be modified on the customer site - and in evaluation phase. Along with an exhaustive set of experiments, in this paper, we demonstrate the suitability of the proposed setup for effective illumination of complex-objects, defined as manufactured items with variable surface characteristics that cannot be determined a priori. Moreover, we discuss the importance of multiple light configurations availability during training and their natural boosting effect which, without the need to modify the system design in evaluation phase, lead to improvements in the overall system performance.
Istituto Italiano di Tecnologia（意大利理工学院）；Huawei Technologies Co., Ltd.（华为技术有限公司）；University of Verona（维罗纳大学）；University of Genova（热那亚大学）
This paper presents a novel setup for automatic visual inspection of cracks in ceramic tile as well as studies the effect of various classifiers and height-varying illumination conditions for this task. The intuition behind this setup is that cracks can be better visualized under specific lighting conditions than others. Our setup, which is designed for field work with constraints in its maximum dimensions, can acquire images for crack detection with multiple lighting conditions using the illumination sources placed at multiple heights. Crack detection is then performed by classifying patches …
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（达尔文人工智能公司）；Moog Inc.（穆格公司）
In electronics manufacturing, solder joint defects are a common problem affecting a variety of printed circuit board components. To identify and correct solder joint defects, the solder joints on a circuit board are typically inspected manually by trained human inspectors, which is a very time-consuming and error-prone process. To improve both inspection efficiency and accuracy, in this work we describe an explainable deep learning-based visual quality inspection system tailored for visual inspection of solder joints in electronics manufacturing environments. At the core of this system is an e…
To address the problem of online automatic inspection of drug liquid bottles in production line, an implantable visual inspection system is designed and the ensemble learning algorithm for detection is proposed based on multi-features fusion. A tunnel structure is designed for visual inspection system, which allows bottles inspection to be automated without changing original
University of Zagreb Faculty of Electrical and Computer Engineering（萨格勒布大学电气工程与计算机学院）
Most visual quality inspections in discrete manufacturing are composed of length, surface, angle or intensity measurements. Those are implemented as end-user configurable inspection tools that should not require an image processing expert to set up. Currently available software solutions providing such capability use a flowchart based programming environment, but do not fully address an inspection flowchart robustness and can require a redefinition of the flowchart if a small variation is introduced. In this paper we propose an acquire-register-analyze image processing pattern designed for dis…
People's visual experiences of the world are easy to carve up and examine along natural language boundaries, e.g., by category labels, attribute labels, etc. However, it is more difficult to elicit detailed visuospatial information about what a person attends to, e.g., the specific shape of a tree. Paying attention to the shapes of things not only feeds into well defined tasks like visual category learning, but it is also what enables us to differentiate similarly named objects and to take on creative visual pursuits, like poetically describing the shape of a thing, or finding shapes in the cl…
Czech Technical University in Prague（捷克技术大学）；IBM Research Zürich（IBM苏黎世研究院）
Labeling images for visual segmentation is a time-consuming task which can be costly, particularly in application domains where labels have to be provided by specialized expert annotators, such as civil engineering. In this paper, we propose to use attribution methods to harness the valuable interactions between expert annotators and the data to be annotated in the case of defect segmentation for visual inspection of civil infrastructures. Concretely, a classifier is trained to detect defects and coupled with an attribution-based method and adversarial climbing to generate and refine segmentat…
Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Costa Rica Institute of Technology（哥斯达黎加理工学院）；Gestalt Automation GmbH（Gestalt Automation有限公司）；Technical University Berlin（柏林工业大学）
Multi-Level Feature Fusion for Continual Learning in Visual Quality Inspection
Technical University of Munich, Institute for Machine Tools（慕尼黑技术大学，机床研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎矿学院，PSL大学，数学形态学中心（CMM））
A Dataset and Baseline for Deep Learning-Based Visual Quality Inspection in Remanufacturing
Institute for Machine Tools and Industrial Management (iwb), Technical University of Munich（慕尼黑工业大学机床与工业管理研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎高科矿业学院，巴黎文理研究大学，数学形态学中心）；Fraunhofer Institute for Casting, Composite and Processing Technology IGCV（弗劳恩霍夫铸造、复合材料与加工技术研究所）
This paper present the mathematical fundaments and experimental study of an algorithm used to find the optimal position for the camera lens to obtain a maximum of details. This information can be further applied to a appropriate system to automatically correct this position. The algorithm is based on the evaluation of a so called resolution function who calculates the maximum of gradient in a certain zone of the image. The paper also presents alternative forms of the function, results of measurements and set up a set of practical rules for the right application of the algorithm.
Attention Modules Improve Image-Level Anomaly Detection for Industrial Inspection: A DifferNet Case Study
Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco（佩南布科联邦大学信息学中心 Voxar 实验室）；Visual Computing Lab, DC, Universidade Federal Rural de Pernambuco（佩南布科联邦农村大学 DC 视觉计算实验室）；Chemnitz University of Technology（开姆尼茨工业大学）
Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual Inspection
National University of Singapore（新加坡国立大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Gongshang University（浙江工商大学）；Wuhan University（武汉大学）
Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；University of Ljubljana, Faculty of Electrical Engineering（卢布尔雅那大学电气工程学院）；Philips Consumer Lifestyle BV（飞利浦消费生活有限公司）；Qlector d.o.o.（Qlector有限公司）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing companies to verify product conformance to the requirements and specifications. Standardized quality control ensures that all the products are evaluated under the same criteria. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing and provided greater data availability. Such data availability has spurred the development of artificial intelligence models, which allow higher degrees of automation and reduced bias when inspecting the products. Furthermore, the increased speed of inspection…
GEOMAR Helmholtz Centre for Ocean Research Kiel（基尔亥姆霍兹海洋研究中心）；Fachhochschule Kiel（基尔应用技术大学）
The majority of Earth's surface lies deep in the oceans, where no surface light reaches. Robots diving down to great depths must bring light sources that create moving illumination patterns in the darkness, such that the same 3D point appears with different color in each image. On top, scattering and attenuation of light in the water makes images appear foggy and typically blueish, the degradation depending on each pixel's distance to its observed seafloor patch, on the local composition of the water and the relative poses and cones of the light sources. Consequently, visual mapping, including…
Bridges are an essential part of the transportation infrastructure and need to be monitored periodically. Visual inspections by dedicated teams have been one of the primary tools in structural health monitoring (SHM) of bridge structures. However, such conventional methods have certain shortcomings. Manual inspections may be challenging in harsh environments and are commonly biased in nature. In the last decade, camera-equipped unmanned aerial vehicles (UAVs) have been widely used for visual inspections; however, the task of automatically extracting useful information from raw images is still …
More than a thousand 8" silicon sensors will be visually inspected to look for anomalies on their surface during the quality control preceding assembly into the High-Granularity Calorimeter for the CMS experiment at CERN. A deep learning-based algorithm that pre-selects potentially anomalous images of the sensor surface in real time has been developed to automate the visual inspection. The anomaly detection is done by an ensemble of independent deep convolutional neural networks: an autoencoder and a classifier. The performance is evaluated on images acquired in production. The pre-selection r…
Software ITC；Technical University Cluj-Napoca（克卢日-纳波卡技术大学）；Faculty of Electronics and Telecommunications（电子与电信学院）
This paper draws a proposal of a set of parameters and methods for accuracy evaluation of visual inspection systems. The case of a monochrome board is treated, but practically all conclusions and methods may be extended for colour acquisition. Basically, the proposed parameters are grouped in five sets as follows:Internal noise;Video ADC cuantisation parameters;Analogue processing section parameters;Dominant frequencies;Synchronisation (lock-in) accuracy. On basis of this set of parameters was developed a software environment, in conjunction with a test signal generator that allows the "test" …
Federal University of Tecnology - Paraná (UTFPR)（巴拉那联邦理工大学）
Automotive manufacturing assembly tasks are built upon visual inspections such as scratch identification on machined surfaces, part identification and selection, etc, which guarantee product and process quality. These tasks can be related to more than one type of vehicle that is produced within the same manufacturing line. Visual inspection was essentially human-led but has recently been supplemented by the artificial perception provided by computer vision systems (CVSs). Despite their relevance, the accuracy of CVSs varies accordingly to environmental settings such as lighting, enclosure and …
University at Buffalo, the State University of New York（纽约州立大学布法罗分校）
Visual inspection is the predominant technique for evaluating the condition of civil infrastructure. The recent advances in unmanned aerial vehicles (UAVs) and artificial intelligence have made the visual inspections faster, safer, and more reliable. Camera-equipped UAVs are becoming the new standard in the industry by collecting massive amounts of visual data for human inspectors. Meanwhile, there has been significant research on autonomous visual inspections using deep learning algorithms, including semantic segmentation. While UAVs can capture high-resolution images of buildings' façades, h…
Adobe Research（Adobe研究院）；Indian Institute of Tech. Dhanbad（印度理工学院（丹巴德））；Université Laval（拉瓦尔大学）
We propose a data-driven learned sky model, which we use for outdoor lighting estimation from a single image. As no large-scale dataset of images and their corresponding ground truth illumination is readily available, we use complementary datasets to train our approach, combining the vast diversity of illumination conditions of SUN360 with the radiometrically calibrated and physically accurate Laval HDR sky database. Our key contribution is to provide a holistic view of both lighting modeling and estimation, solving both problems end-to-end. From a test image, our method can directly estimate …
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；National Technical University of Athens（雅典国立技术大学）；Philips Consumer Lifestyle BV（飞利浦消费者生活方式公司）；Qlector d.o.o.（Qlector公司）
Quality control is a crucial activity performed by manufacturing companies to ensure their products conform to the requirements and specifications. The introduction of artificial intelligence models enables to automate the visual quality inspection, speeding up the inspection process and ensuring all products are evaluated under the same criteria. In this research, we compare supervised and unsupervised defect detection techniques and explore data augmentation techniques to mitigate the data imbalance in the context of automated visual inspection. Furthermore, we use Generative Adversarial Net…
DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenes
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（整合视觉与语言实验室，电气工程学院，韩国科学技术院（KAIST））
IBM China Research Lab（IBM中国研究院）；Shanghai Jiao Tong University（上海交通大学）
With the rapid growth of display devices, quality inspection via machine vision technology has become increasingly important for flat-panel displays (FPD) industry. This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. The system is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture. A hybrid application of conventional computer vision technique and the latest deep convolutional neural network (DCNN) leads to an integrated defec…
University of Ljubljana（卢布尔雅那大学）；Jožef Stefan International Postgraduate School（约热夫·斯特凡国际研究生院）；Philips Consumer Lifestyle BV（飞利浦消费生活方式有限公司）；Qlector d.o.o.（Qlector有限责任公司）；Jožef Stefan Institute（约热夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing enterprises to ensure products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. In this research, we compare three active learning approaches and five machine learning algorithms applied to visual defect inspection with real-world data provided by Philips Consumer Lifes…
A critical aspect in the manufacturing process is the visual quality inspection of manufactured components for defects and flaws. Human-only visual inspection can be very time-consuming and laborious, and is a significant bottleneck especially for high-throughput manufacturing scenarios. Given significant advances in the field of deep learning, automated visual quality inspection can lead to highly efficient and reliable detection of defects and flaws during the manufacturing process. However, deep learning-driven visual inspection methods often necessitate significant computational resources,…
Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Recent advances in the industrial inspection of textured surfaces-in the form of visual inspection-have made such inspections possible for efficient, flexible manufacturing systems. We propose an unsupervised feature memory rearrangement network (FMR-Net) to accurately detect various textural defects simultaneously. Consistent with mainstream methods, we adopt the idea of background reconstruction; however, we innovatively utilize artificial synthetic defects to enable the model to recognize anomalies, while traditional wisdom relies only on defect-free samples. First, we employ an encoding mo…
Communication channel established from a display to a device's camera is known as visual channel, and it is helpful in securing key exchange protocol. In this paper, we study how visual channel can be exploited by a network terminal and mobile device to jointly verify information in an interactive session, and how such information can be jointly presented in a user-friendly manner, taking into account that the mobile device can only capture and display a small region, and the user may only want to authenticate selective regions-of-interests. Motivated by applications in Kiosk computing and mul…
Politecnico di Milano（米兰理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Iran University of Science and Technology（伊朗科学技术大学）；Swiss Federal Institute of Technology at Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
A current trend in industries such as semiconductors and foundry is to shift their visual inspection processes to Automatic Visual Inspection (AVI) systems, to reduce their costs, mistakes, and dependency on human experts. This paper proposes a two-staged fault diagnosis framework for AVI systems. In the first stage, a generation model is designed to synthesize new samples based on real samples. The proposed augmentation algorithm extracts objects from the real samples and blends them randomly, to generate new samples and enhance the performance of the image processor. In the second stage, an …
The detection and the quantification of anomalies in image data are critical tasks in industrial scenes such as detecting micro scratches on product. In recent years, due to the difficulty of defining anomalies and the limit of correcting their labels, research on unsupervised anomaly detection using generative models has attracted attention. Generally, in those studies, only normal images are used for training to model the distribution of normal images. The model measures the anomalies in the target images by reproducing the most similar images and scoring image patches indicating their fit t…
Human in the AI loop via xAI and Active Learning for Visual Inspection
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Qlector d.o.o.（Qlector公司）；University of Applied Sciences and Arts of Southern Switzerland（瑞士南部应用科学与艺术大学）；Ubitech Ltd.（Ubitech有限公司）；University of Ljubljana（卢布尔雅那大学）；University of West Attica（西阿提卡大学）；University of Groningen（格罗宁根大学）
本文提出结合主动学习与可解释人工智能，将人类纳入视觉检测的 AI 回路，并综述工业 5.0 人机协作及 STAR 项目相关成果。
The traditional mode of recording faults in heavy factory equipment has been via hand marked inspection sheets, wherein a machine engineer manually marks the faulty machine regions on a paper outline of the machine. Over the years, millions of such inspection sheets have been recorded and the data within these sheets has remained inaccessible. However, with industries going digital and waking up to the potential value of fault data for machine health monitoring, there is an increased impetus towards digitization of these hand marked inspection records. To target this digitization, we propose a…
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；University of Petroleum and Energy Studies（石油与能源研究大学）
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component's contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to auto…
Institute for Infocomm Research (IR), A*STAR（新加坡科技研究局资讯通信研究院）；Nanyang Technological University (NTU)（南洋理工大学）；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
Advances in Unmanned Aerial Vehicle (UAV) opens venues for application such as tunnel inspection. Owing to its versatility to fly inside the tunnels, it can quickly identify defects and potential problems related to safety. However, long tunnels, especially with repetitive or uniform structures pose a significant problem for UAV navigation. Furthermore, post-processing visual data from the camera mounted on the UAV is required to generate useful information for the inspection task. In this work, we design a UAV with a single rotating camera to accomplish the task. Compared to other platforms, …
Anomaly detection has recently seen great progress in the field of visual inspection. More specifically, the use of classical outlier detection techniques on features extracted by deep pre-trained neural networks have been shown to deliver remarkable performances on the MVTec Anomaly Detection (MVTec AD) dataset. However, like most other anomaly detection strategies, these pre-trained methods assume all training data to be normal. As a consequence, they cannot be considered as fully unsupervised. There exists to our knowledge no work studying these pre-trained methods under fully unsupervised …
