This paper draws a proposal of a set of parameters and methods for accuracy evaluation of visual inspection systems. The case of a monochrome board is treated, but practically all conclusions and methods may be extended for colour acquisition. Basically, the proposed parameters are grouped in five sets as follows:Internal noise;Video ADC cuantisation parameters;Analogue processing section parameters;Dominant frequencies;Synchronisation (lock-in) accuracy. On basis of this set of parameters was developed a software environment, in conjunction with a test signal generator that allows the "test" images. The paper also presents conclusions of evaluation for two types of video acquisition boards
Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute（弗劳恩霍夫海因里希·赫兹通信研究所）；Zentrum für Bild- und Signalverarbeitung（图像与信号处理中心）；Humboldt University Berlin（柏林洪堡大学）
The assessment of sewer pipe systems is a highly important, but at the same time cumbersome and error-prone task. We introduce an innovative system based on single-shot structured light modules that facilitates the detection and classification of spatial defects like jutting intrusions, spallings, or misaligned joints. This system creates highly accurate 3D measurements with sub-millimeter resolution of pipe surfaces and fuses them into a holistic 3D model. The benefit of such a holistic 3D model is twofold: on the one hand, it facilitates the accurate manual sewer pipe assessment, on the othe…
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（达尔文人工智能公司）；Moog Inc.（穆格公司）
In electronics manufacturing, solder joint defects are a common problem affecting a variety of printed circuit board components. To identify and correct solder joint defects, the solder joints on a circuit board are typically inspected manually by trained human inspectors, which is a very time-consuming and error-prone process. To improve both inspection efficiency and accuracy, in this work we describe an explainable deep learning-based visual quality inspection system tailored for visual inspection of solder joints in electronics manufacturing environments. At the core of this system is an e…
To address the problem of online automatic inspection of drug liquid bottles in production line, an implantable visual inspection system is designed and the ensemble learning algorithm for detection is proposed based on multi-features fusion. A tunnel structure is designed for visual inspection system, which allows bottles inspection to be automated without changing original
University of Zagreb Faculty of Electrical and Computer Engineering（萨格勒布大学电气工程与计算机学院）
Most visual quality inspections in discrete manufacturing are composed of length, surface, angle or intensity measurements. Those are implemented as end-user configurable inspection tools that should not require an image processing expert to set up. Currently available software solutions providing such capability use a flowchart based programming environment, but do not fully address an inspection flowchart robustness and can require a redefinition of the flowchart if a small variation is introduced. In this paper we propose an acquire-register-analyze image processing pattern designed for dis…
People's visual experiences of the world are easy to carve up and examine along natural language boundaries, e.g., by category labels, attribute labels, etc. However, it is more difficult to elicit detailed visuospatial information about what a person attends to, e.g., the specific shape of a tree. Paying attention to the shapes of things not only feeds into well defined tasks like visual category learning, but it is also what enables us to differentiate similarly named objects and to take on creative visual pursuits, like poetically describing the shape of a thing, or finding shapes in the cl…
Czech Technical University in Prague（捷克技术大学）；IBM Research Zürich（IBM苏黎世研究院）
Labeling images for visual segmentation is a time-consuming task which can be costly, particularly in application domains where labels have to be provided by specialized expert annotators, such as civil engineering. In this paper, we propose to use attribution methods to harness the valuable interactions between expert annotators and the data to be annotated in the case of defect segmentation for visual inspection of civil infrastructures. Concretely, a classifier is trained to detect defects and coupled with an attribution-based method and adversarial climbing to generate and refine segmentat…
More than a thousand 8" silicon sensors will be visually inspected to look for anomalies on their surface during the quality control preceding assembly into the High-Granularity Calorimeter for the CMS experiment at CERN. A deep learning-based algorithm that pre-selects potentially anomalous images of the sensor surface in real time has been developed to automate the visual inspection. The anomaly detection is done by an ensemble of independent deep convolutional neural networks: an autoencoder and a classifier. The performance is evaluated on images acquired in production. The pre-selection r…
Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；University of Ljubljana, Faculty of Electrical Engineering（卢布尔雅那大学电气工程学院）；Philips Consumer Lifestyle BV（飞利浦消费生活有限公司）；Qlector d.o.o.（Qlector有限公司）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing companies to verify product conformance to the requirements and specifications. Standardized quality control ensures that all the products are evaluated under the same criteria. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing and provided greater data availability. Such data availability has spurred the development of artificial intelligence models, which allow higher degrees of automation and reduced bias when inspecting the products. Furthermore, the increased speed of inspection…
The design of an automatic visual inspection system is usually performed in two stages. While the first stage consists in selecting the most suitable hardware setup for highlighting most effectively the defects on the surface to be inspected, the second stage concerns the development of algorithmic solutions to exploit the potentials offered by the collected data. In this paper, first, we present a novel illumination setup embedding four illumination configurations to resemble diffused, dark-field, and front lighting techniques. Second, we analyze the contributions brought by deploying the pro…
IBM China Research Lab（IBM中国研究院）；Shanghai Jiao Tong University（上海交通大学）
With the rapid growth of display devices, quality inspection via machine vision technology has become increasingly important for flat-panel displays (FPD) industry. This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. The system is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture. A hybrid application of conventional computer vision technique and the latest deep convolutional neural network (DCNN) leads to an integrated defec…
University of Ljubljana（卢布尔雅那大学）；Jožef Stefan International Postgraduate School（约热夫·斯特凡国际研究生院）；Philips Consumer Lifestyle BV（飞利浦消费生活方式有限公司）；Qlector d.o.o.（Qlector有限责任公司）；Jožef Stefan Institute（约热夫·斯特凡研究所）
Quality control is a key activity performed by manufacturing enterprises to ensure products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled an increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. In this research, we compare three active learning approaches and five machine learning algorithms applied to visual defect inspection with real-world data provided by Philips Consumer Lifes…
A critical aspect in the manufacturing process is the visual quality inspection of manufactured components for defects and flaws. Human-only visual inspection can be very time-consuming and laborious, and is a significant bottleneck especially for high-throughput manufacturing scenarios. Given significant advances in the field of deep learning, automated visual quality inspection can lead to highly efficient and reliable detection of defects and flaws during the manufacturing process. However, deep learning-driven visual inspection methods often necessitate significant computational resources,…
Zero-Shot Multi-Criteria Visual Quality Inspection for Semi-Controlled Industrial Environments via Real-Time 3D Digital Twin Simulation
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；Costa Rica Institute of Technology（哥斯达黎加理工学院）；Gestalt Automation GmbH（Gestalt Automation有限公司）；Technical University Berlin（柏林工业大学）
Federal University of Tecnology - Paraná (UTFPR)（巴拉那联邦理工大学）
Automotive manufacturing assembly tasks are built upon visual inspections such as scratch identification on machined surfaces, part identification and selection, etc, which guarantee product and process quality. These tasks can be related to more than one type of vehicle that is produced within the same manufacturing line. Visual inspection was essentially human-led but has recently been supplemented by the artificial perception provided by computer vision systems (CVSs). Despite their relevance, the accuracy of CVSs varies accordingly to environmental settings such as lighting, enclosure and …
University at Buffalo, the State University of New York（纽约州立大学布法罗分校）
Visual inspection is the predominant technique for evaluating the condition of civil infrastructure. The recent advances in unmanned aerial vehicles (UAVs) and artificial intelligence have made the visual inspections faster, safer, and more reliable. Camera-equipped UAVs are becoming the new standard in the industry by collecting massive amounts of visual data for human inspectors. Meanwhile, there has been significant research on autonomous visual inspections using deep learning algorithms, including semantic segmentation. While UAVs can capture high-resolution images of buildings' façades, h…
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；National Technical University of Athens（雅典国立技术大学）；Philips Consumer Lifestyle BV（飞利浦消费者生活方式公司）；Qlector d.o.o.（Qlector公司）
Quality control is a crucial activity performed by manufacturing companies to ensure their products conform to the requirements and specifications. The introduction of artificial intelligence models enables to automate the visual quality inspection, speeding up the inspection process and ensuring all products are evaluated under the same criteria. In this research, we compare supervised and unsupervised defect detection techniques and explore data augmentation techniques to mitigate the data imbalance in the context of automated visual inspection. Furthermore, we use Generative Adversarial Net…
Multi-Level Feature Fusion for Continual Learning in Visual Quality Inspection
Technical University of Munich, Institute for Machine Tools（慕尼黑技术大学，机床研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎矿学院，PSL大学，数学形态学中心（CMM））
Human in the AI loop via xAI and Active Learning for Visual Inspection
Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Qlector d.o.o.（Qlector公司）；University of Applied Sciences and Arts of Southern Switzerland（瑞士南部应用科学与艺术大学）；Ubitech Ltd.（Ubitech有限公司）；University of Ljubljana（卢布尔雅那大学）；University of West Attica（西阿提卡大学）；University of Groningen（格罗宁根大学）
本文提出结合主动学习与可解释人工智能，将人类纳入视觉检测的 AI 回路，并综述工业 5.0 人机协作及 STAR 项目相关成果。
Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）
Recent advances in the industrial inspection of textured surfaces-in the form of visual inspection-have made such inspections possible for efficient, flexible manufacturing systems. We propose an unsupervised feature memory rearrangement network (FMR-Net) to accurately detect various textural defects simultaneously. Consistent with mainstream methods, we adopt the idea of background reconstruction; however, we innovatively utilize artificial synthetic defects to enable the model to recognize anomalies, while traditional wisdom relies only on defect-free samples. First, we employ an encoding mo…
Communication channel established from a display to a device's camera is known as visual channel, and it is helpful in securing key exchange protocol. In this paper, we study how visual channel can be exploited by a network terminal and mobile device to jointly verify information in an interactive session, and how such information can be jointly presented in a user-friendly manner, taking into account that the mobile device can only capture and display a small region, and the user may only want to authenticate selective regions-of-interests. Motivated by applications in Kiosk computing and mul…
A Dataset and Baseline for Deep Learning-Based Visual Quality Inspection in Remanufacturing
Institute for Machine Tools and Industrial Management (iwb), Technical University of Munich（慕尼黑工业大学机床与工业管理研究所）；MINES Paris, PSL University, Centre for Mathematical Morphology (CMM)（巴黎高科矿业学院，巴黎文理研究大学，数学形态学中心）；Fraunhofer Institute for Casting, Composite and Processing Technology IGCV（弗劳恩霍夫铸造、复合材料与加工技术研究所）
The traditional mode of recording faults in heavy factory equipment has been via hand marked inspection sheets, wherein a machine engineer manually marks the faulty machine regions on a paper outline of the machine. Over the years, millions of such inspection sheets have been recorded and the data within these sheets has remained inaccessible. However, with industries going digital and waking up to the potential value of fault data for machine health monitoring, there is an increased impetus towards digitization of these hand marked inspection records. To target this digitization, we propose a…
This paper present the mathematical fundaments and experimental study of an algorithm used to find the optimal position for the camera lens to obtain a maximum of details. This information can be further applied to a appropriate system to automatically correct this position. The algorithm is based on the evaluation of a so called resolution function who calculates the maximum of gradient in a certain zone of the image. The paper also presents alternative forms of the function, results of measurements and set up a set of practical rules for the right application of the algorithm.
Politecnico di Milano（米兰理工大学）；University of British Columbia（不列颠哥伦比亚大学）；Iran University of Science and Technology（伊朗科学技术大学）；Swiss Federal Institute of Technology at Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
A current trend in industries such as semiconductors and foundry is to shift their visual inspection processes to Automatic Visual Inspection (AVI) systems, to reduce their costs, mistakes, and dependency on human experts. This paper proposes a two-staged fault diagnosis framework for AVI systems. In the first stage, a generation model is designed to synthesize new samples based on real samples. The proposed augmentation algorithm extracts objects from the real samples and blends them randomly, to generate new samples and enhance the performance of the image processor. In the second stage, an …
Attention Modules Improve Image-Level Anomaly Detection for Industrial Inspection: A DifferNet Case Study
Voxar Labs, Centro de Informática, Universidade Federal de Pernambuco（佩南布科联邦大学信息学中心 Voxar 实验室）；Visual Computing Lab, DC, Universidade Federal Rural de Pernambuco（佩南布科联邦农村大学 DC 视觉计算实验室）；Chemnitz University of Technology（开姆尼茨工业大学）
Prototypical Cross-domain Knowledge Transfer for Cervical Dysplasia Visual Inspection
National University of Singapore（新加坡国立大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；Northwestern Polytechnical University（西北工业大学）；Zhejiang Gongshang University（浙江工商大学）；Wuhan University（武汉大学）
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；University of Petroleum and Energy Studies（石油与能源研究大学）
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component's contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to auto…
ONERA, Université Paris Saclay（法国航空航天实验室（ONERA），巴黎萨克雷大学）；LIRMM, Université de Montpellier, CNRS（蒙彼利埃计算机、机器人与微电子实验室，蒙彼利埃大学，法国国家科学研究中心）；DRASSM, Ministry of Culture（法国文化部水下与海底考古研究局）
This paper presents a new underwater dataset acquired from a visual-inertial-pressure acquisition system and meant to be used to benchmark visual odometry, visual SLAM and multi-sensors SLAM solutions. The dataset is publicly available and contains ground-truth trajectories for evaluation.
Bridges are an essential part of the transportation infrastructure and need to be monitored periodically. Visual inspections by dedicated teams have been one of the primary tools in structural health monitoring (SHM) of bridge structures. However, such conventional methods have certain shortcomings. Manual inspections may be challenging in harsh environments and are commonly biased in nature. In the last decade, camera-equipped unmanned aerial vehicles (UAVs) have been widely used for visual inspections; however, the task of automatically extracting useful information from raw images is still …
Anomaly detection has recently seen great progress in the field of visual inspection. More specifically, the use of classical outlier detection techniques on features extracted by deep pre-trained neural networks have been shown to deliver remarkable performances on the MVTec Anomaly Detection (MVTec AD) dataset. However, like most other anomaly detection strategies, these pre-trained methods assume all training data to be normal. As a consequence, they cannot be considered as fully unsupervised. There exists to our knowledge no work studying these pre-trained methods under fully unsupervised …
Institute for Infocomm Research (IR), A*STAR（新加坡科技研究局资讯通信研究院）；Nanyang Technological University (NTU)（南洋理工大学）；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
Advances in Unmanned Aerial Vehicle (UAV) opens venues for application such as tunnel inspection. Owing to its versatility to fly inside the tunnels, it can quickly identify defects and potential problems related to safety. However, long tunnels, especially with repetitive or uniform structures pose a significant problem for UAV navigation. Furthermore, post-processing visual data from the camera mounted on the UAV is required to generate useful information for the inspection task. In this work, we design a UAV with a single rotating camera to accomplish the task. Compared to other platforms, …
DIP-R1: Deep Inspection and Perception with RL Looking Through and Understanding Complex Scenes
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)（整合视觉与语言实验室，电气工程学院，韩国科学技术院（KAIST））
JR East Information Systems Company（东日本旅客铁道信息系统公司）；East Japan Railway Company（东日本旅客铁道公司）
The external visual inspections of rolling stock's underfloor equipment are currently being performed via human visual inspection. In this study, we attempt to partly automate visual inspection by investigating anomaly inspection algorithms that use image processing technology. As the railroad maintenance studies tend to have little anomaly data, unsupervised learning methods are usually preferred for anomaly detection; however, training cost and accuracy is still a challenge. Additionally, a researcher created anomalous images from normal images by adding noise, etc., but the anomalous target…
Visual Question Generation (VQG) is a task to generate questions from images. When humans ask questions about an image, their goal is often to acquire some new knowledge. However, existing studies on VQG have mainly addressed question generation from answers or question categories, overlooking the objectives of knowledge acquisition. To introduce a knowledge acquisition perspective into VQG, we constructed a novel knowledge-aware VQG dataset called K-VQG. This is the first large, humanly annotated dataset in which questions regarding images are tied to structured knowledge. We also developed a…
The use of deep features coming from pre-trained neural networks for unsupervised anomaly detection purposes has recently gathered momentum in the computer vision field. In particular, industrial inspection applications can take advantage of such features, as demonstrated by the multiple successes of related methods on the MVTec Anomaly Detection (MVTec AD) dataset. These methods make use of neural networks pre-trained on auxiliary classification tasks such as ImageNet. However, to our knowledge, no comparative study of robustness to the low data regimes between these approaches has been condu…
Jožef Stefan International Postgraduate School（约瑟夫·斯特凡国际研究生院）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）；Qlector d.o.o.（Qlector有限责任公司）；Faculty of Electrical Engineering, University of Ljubljana（卢布尔雅那大学电气工程学院）；Philips Consumer Lifestyle BV（飞利浦消费生活方式公司）
Quality control is a crucial activity performed by manufacturing enterprises to ensure that their products meet quality standards and avoid potential damage to the brand's reputation. The decreased cost of sensors and connectivity enabled increasing digitalization of manufacturing. In addition, artificial intelligence enables higher degrees of automation, reducing overall costs and time required for defect inspection. This research compares three active learning approaches, having single and multiple oracles, to visual inspection. Six new metrics are proposed to assess the quality of calibrati…
The detection and the quantification of anomalies in image data are critical tasks in industrial scenes such as detecting micro scratches on product. In recent years, due to the difficulty of defining anomalies and the limit of correcting their labels, research on unsupervised anomaly detection using generative models has attracted attention. Generally, in those studies, only normal images are used for training to model the distribution of normal images. The model measures the anomalies in the target images by reproducing the most similar images and scoring image patches indicating their fit t…
Method of UAV Inspection of Photovoltaic Modules Using Thermal and RGB Data Fusion
Khmelnytskyi National University（赫梅利尼茨基国立大学）；Department of Informatics and Teleinformatics（信息学与电信学系）；Kazimierz Pulaski University of Ra-dom（卡齐米日·波尼亚图夫斯基大学）；National Academy of the State Border Service of Ukraine named after Bogdan Khmelnitsky（以博丹·赫梅利尼茨基命名的乌克兰国家边境服务学院）
