Automatic crack detection is a critical task that has the potential to drastically reduce labor-intensive building and road inspections currently being done manually. Recent studies in this field have significantly improved the detection accuracy. However, the methods often heavily rely on costly annotation processes. In addition, to handle a wide variety of target domains, new batches of annotations are usually required for each new environment. This makes the data annotation cost a significant bottleneck when deploying crack detection systems in real life. To resolve this issue, we formulate the crack detection problem as a weakly-supervised problem and propose a two-branched framework. By combining predictions of a supervised model trained on low quality annotations with predictions based on pixel brightness, our framework is less affected by the annotation quality. Experimental results show that the proposed framework retains high detection accuracy even when provided with low quality annotations. Implementation of the proposed framework is publicly available at https://github.com/hitachi-rd-cv/weakly-sup-crackdet.
Pixel-level crack segmentation is widely studied due to its high impact on building and road inspections. While recent studies have made significant improvements in accuracy, they typically heavily depend on pixel-level crack annotations, which are time-consuming to obtain. In earlier work, we proposed to reduce the annotation cost bottleneck by reformulating the crack segmentation problem as a weakly-supervised problem -- i.e. the annotation process is expedited by sacrificing the annotation quality. The loss in annotation quality was remedied by refining the inference with per-pixel brightne…
Indian Institute of Technology, Madras（印度理工学院马德拉斯分校）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Rizzo International, Inc.（里佐国际公司）
The detection of cracks is a crucial task in monitoring structural health and ensuring structural safety. The manual process of crack detection is time-consuming and subjective to the inspectors. Several researchers have tried tackling this problem using traditional Image Processing or learning-based techniques. However, their scope of work is limited to detecting cracks on a single type of surface (walls, pavements, glass, etc.). The metrics used to evaluate these methods are also varied across the literature, making it challenging to compare techniques. This paper addresses these problems by…
Graduate School of Information Sciences, Tohoku University（东北大学信息科学研究生院）；RIKEN Center for AIP（理化学研究所人工智能中心）
In this study, we consider the problem of detecting cracks from the image of a concrete surface for automated inspection of infrastructure, such as bridges. Its overall accuracy is determined by how accurately thin cracks with sub-pixel widths can be detected. Our interest is in making it possible to detect cracks close to the limit of thinness if it can be defined. Toward this end, we first propose a method for training a CNN to make it detect cracks more accurately than humans while training them on human-annotated labels. To achieve this seemingly impossible goal, we intentionally lower the…
WP-CrackNet: A Collaborative Adversarial Learning Framework for End-to-End Weakly-Supervised Road Crack Detection
College of Electronic and Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University（电子信息工程学院，智能自主系统上海研究院，同济大学）；School of Internet of Things Engineering, Jiangnan University（物联网工程学院，江南大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（电气电子工程学院，南洋理工大学）
Improving Post-Earthquake Crack Detection using Semi-Synthetic Generated Images
Department of Electrical, Computer and Biomedical Engineering, University of Pavia（电气、计算机与生物医学工程系，帕维亚大学）；EUCENTRE Foundation, European Centre for Training and Research in Earthquake Engineering（欧洲地震工程培训与研究中心基金会）
Friedrich-Alexander University Erlangen-Nuremberg（弗里德里希-亚历山大-埃尔兰根-纽伦堡大学）
Photovoltaic is one of the most important renewable energy sources for dealing with world-wide steadily increasing energy consumption. This raises the demand for fast and scalable automatic quality management during production and operation. However, the detection and segmentation of cracks on electroluminescence (EL) images of mono- or polycrystalline solar modules is a challenging task. In this work, we propose a weakly supervised learning strategy that only uses image-level annotations to obtain a method that is capable of segmenting cracks on EL images of solar cells. We use a modified Res…
Unity-Drive Technology Inc（深圳一清科技有限公司）；Hong Kong University of Science and Technology（香港科技大学）；National Engineering Research Center of Road Maintenance Technologies（道路养护技术国家工程研究中心）；University of Bristol（布里斯托大学）
Crack is one of the most common road distresses which may pose road safety hazards. Generally, crack detection is performed by either certified inspectors or structural engineers. This task is, however, time-consuming, subjective and labor-intensive. In this paper, we propose a novel road crack detection algorithm based on deep learning and adaptive image segmentation. Firstly, a deep convolutional neural network is trained to determine whether an image contains cracks or not. The images containing cracks are then smoothed using bilateral filtering, which greatly minimizes the number of noisy …
Silesian University of Technology（西里西亚理工大学）；Chung-Ang University（中央大学）；Polish Academy of Sciences（波兰科学院）
Monitoring the technical condition of infrastructure is a crucial element to its maintenance. Currently applied methods are outdated, labour-intensive and inaccurate. At the same time, the latest methods using Artificial Intelligence techniques are severely limited in their application due to two main factors -- labour-intensive gathering of new datasets and high demand for computing power. We propose to utilize custom made framework -- KrakN, to overcome these limiting factors. It enables the development of unique infrastructure defects detectors on digital images, achieving the accuracy of a…
From Classification to Segmentation with Explainable AI: A Study on Crack Detection and Growth Monitoring
Intelligent Maintenance and Operations Systems (IMOS), EPFL（智能维护与运营系统研究所（IMOS），洛桑联邦理工学院）；Environmental Computational Science and Earth Observation (ECEO), EPFL（环境计算科学与地球观测研究所（ECEO），洛桑联邦理工学院）
Recently, weakly-supervised image segmentation using weak annotations like scribbles has gained great attention, since such annotations are much easier to obtain compared to time-consuming and label-intensive labeling at the pixel/voxel level. However, because scribbles lack structure information of region of interest (ROI), existing scribble-based methods suffer from poor boundary localization. Furthermore, most current methods are designed for 2D image segmentation, which do not fully leverage the volumetric information if directly applied to image slices. In this paper, we propose a scribbl…
University of Ljubljana, Faculty of Computer and Information Science（卢布尔雅那大学计算机与信息科学学院）
Robust maritime obstacle detection is critical for safe navigation of autonomous boats and timely collision avoidance. The current state-of-the-art is based on deep segmentation networks trained on large datasets. However, per-pixel ground truth labeling of such datasets is labor-intensive and expensive. We propose a new scaffolding learning regime (SLR) that leverages weak annotations consisting of water edges, the horizon location, and obstacle bounding boxes to train segmentation-based obstacle detection networks, thereby reducing the required ground truth labeling effort by a factor of twe…
Nanyang Technological University（南洋理工大学）；NVIDIA AI Technology Center（英伟达AI技术中心）；Amazon Rekognition（亚马逊Rekognition）
Object detection is one of the major problems in computer vision, and has been extensively studied. Most of the existing detection works rely on labor-intensive supervision, such as ground truth bounding boxes of objects or at least image-level annotations. On the contrary, we propose an object detection method that does not require any form of human annotation on target tasks, by exploiting freely available web images. In order to facilitate effective knowledge transfer from web images, we introduce a multi-instance multi-label domain adaption learning framework with two key innovations. Firs…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；NVIDIA（英伟达）；University of Oregon（俄勒冈大学）
Tracking segmentation masks of multiple instances has been intensively studied, but still faces two fundamental challenges: 1) the requirement of large-scale, frame-wise annotation, and 2) the complexity of two-stage approaches. To resolve these challenges, we introduce a novel semi-supervised framework by learning instance tracking networks with only a labeled image dataset and unlabeled video sequences. With an instance contrastive objective, we learn an embedding to discriminate each instance from the others. We show that even when only trained with images, the learned feature representatio…
Automated Annotation of Shearographic Measurements Enabling Weakly Supervised Defect Detection
Trier University of Applied Sciences（特里尔应用科学大学）；RPTU University Kaiserslautern-Landau（凯撒斯劳滕-兰道大学）；University of Luxembourg（卢森堡大学）；Fraunhofer Institute for Industrial Mathematics ITWM（弗劳恩霍夫工业数学研究所ITWM）
Shanghai Jiao Tong University（上海交通大学）；University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）；Peking Union Medical College Hospital（北京协和医院）
The segmentation of coronary arteries in X-ray angiograms by convolutional neural networks (CNNs) is promising yet limited by the requirement of precisely annotating all pixels in a large number of training images, which is extremely labor-intensive especially for complex coronary trees. To alleviate the burden on the annotator, we propose a novel weakly supervised training framework that learns from noisy pseudo labels generated from automatic vessel enhancement, rather than accurate labels obtained by fully manual annotation. A typical self-paced learning scheme is used to make the training …
Nanjing University of Science and Technology（南京理工大学）
Video anomaly detection (VAD) is currently a challenging task due to the complexity of anomaly as well as the lack of labor-intensive temporal annotations. In this paper, we propose an end-to-end Global Information Guided (GIG) anomaly detection framework for anomaly detection using the video-level annotations (i.e., weak labels). We propose to first mine the global pattern cues by leveraging the weak labels in a GIG module. Then we build a spatial reasoning module to measure the relevance between vectors in spatial domain with the global cue vectors, and select the most related feature vector…
PM-VIS: High-Performance Box-Supervised Video Instance Segmentation
School of College of Computer Science, Nanjing Audit University（南京审计大学计算机学院）；Stony Brook University（石溪大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Lucid Motors（Lucid Motors（ Lucid汽车公司））
Estimating 3D hand pose from 2D images is a difficult, inverse problem due to the inherent scale and depth ambiguities. Current state-of-the-art methods train fully supervised deep neural networks with 3D ground-truth data. However, acquiring 3D annotations is expensive, typically requiring calibrated multi-view setups or labor intensive manual annotations. While annotations of 2D keypoints are much easier to obtain, how to efficiently leverage such weakly-supervised data to improve the task of 3D hand pose prediction remains an important open question. The key difficulty stems from the fact t…
PC-Seg: Progressive Cross-View Consistency for 3D OCT Segmentation from Sparse 2D Annotations
Graduate School of Information Sciences, Tohoku University（东北大学信息科学研究生院）；Department of Ophthalmology, Graduate School of Medicine, Tohoku University（东北大学医学研究生院眼科系）；Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust（摩尔菲尔德眼科医院NHS基金会信托基金NIHR生物医学研究中心）；UCL Hawkes Institute, University College London（伦敦大学学院UCL霍克斯研究所）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Multimodal Urban Tree Detection from Satellite and Street-Level Imagery via Annotation-Efficient Deep Learning Strategies
Department of Computer Science, University of California, Davis（加州大学戴维斯分校计算机科学系）；Department of Biological and Agricultural Engineering, University of California, Davis（加州大学戴维斯分校生物与农业工程系）
University of Ljubljana（卢布尔雅那大学）；Faculty of Computer and Information Science（计算机与信息科学学院）
Coastal water autonomous boats rely on robust perception methods for obstacle detection and timely collision avoidance. The current state-of-the-art is based on deep segmentation networks trained on large datasets. Per-pixel ground truth labeling of such datasets, however, is labor-intensive and expensive. We observe that far less information is required for practical obstacle avoidance - the location of water edge on static obstacles like shore and approximate location and bounds of dynamic obstacles in the water is sufficient to plan a reaction. We propose a new scaffolding learning regime (…
Image annotation is one essential prior step to enable data-driven algorithms. In medical imaging, having large and reliably annotated data sets is crucial to recognize various diseases robustly. However, annotator performance varies immensely, thus impacts model training. Therefore, often multiple annotators should be employed, which is however expensive and resource-intensive. Hence, it is desirable that users should annotate unseen data and have an automated system to unobtrusively rate their performance during this process. We examine such a system based on whole slide images (WSIs) showin…
Columbia University（哥伦比亚大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Imperial College London（帝国理工学院）；Institut Polytechnique de Paris（巴黎理工学院）
Automatically identifying the structural substrates underlying cardiac abnormalities can potentially provide real-time guidance for interventional procedures. With the knowledge of cardiac tissue substrates, the treatment of complex arrhythmias such as atrial fibrillation and ventricular tachycardia can be further optimized by detecting arrhythmia substrates to target for treatment (i.e., adipose) and identifying critical structures to avoid. Optical coherence tomography (OCT) is a real-time imaging modality that aids in addressing this need. Existing approaches for cardiac image analysis main…
Leverage Weakly Annotation to Pixel-wise Annotation via Zero-shot Segment Anything Model for Molecular-empowered Learning
Data Science Institute, Vanderbilt University（范德堡大学数据科学研究所）；Department of Computer Science, Vanderbilt University（范德堡大学计算机科学系）；NVIDIA Corporation（英伟达公司）；Department of Electrical and Computer Engineering, Vanderbilt University Medical Center（范德堡大学医学中心电气与计算机工程系）；Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center（范德堡大学医学中心病理学、微生物学与免疫学系）
Efficiently Collecting Training Dataset for 2D Object Detection by Online Visual Feedback
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology（独立行政法人产业技术综合研究所）；Nara Institute of Science and Technology (NAIST)（奈良科学技术研究所）
College of Electronics and Information Engineering, Sichuan University（四川大学电子信息学院）；School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Computer Science, The University of Adelaide（阿德莱德大学计算机学院）
Most existing crowd counting methods require object location-level annotation, i.e., placing a dot at the center of an object. While being simpler than the bounding-box or pixel-level annotation, obtaining this annotation is still labor-intensive and time-consuming especially for images with highly crowded scenes. On the other hand, weaker annotations that only know the total count of objects can be almost effortless in many practical scenarios. Thus, it is desirable to develop a learning method that can effectively train models from count-level annotations. To this end, this paper studies the…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究院）
Supervised or weakly supervised methods for phrase localization (textual grounding) either rely on human annotations or some other supervised models, e.g., object detectors. Obtaining these annotations is labor-intensive and may be difficult to scale in practice. We propose to leverage recent advances in contrastive language-vision models, CLIP, pre-trained on image and caption pairs collected from the internet. In its original form, CLIP only outputs an image-level embedding without any spatial resolution. We adapt CLIP to generate high-resolution spatial feature maps. Importantly, we can ext…
As a fine-grained and local expression behavior measurement, facial action unit (FAU) analysis (e.g., detection and intensity estimation) has been documented for its time-consuming, labor-intensive, and error-prone annotation. Thus a long-standing challenge of FAU analysis arises from the data scarcity of manual annotations, limiting the generalization ability of trained models to a large extent. Amounts of previous works have made efforts to alleviate this issue via semi/weakly supervised methods and extra auxiliary information. However, these methods still require domain knowledge and have n…
