This paper focuses on a novel and challenging detection scenario: A majority of true objects/instances is unlabeled in the datasets, so these missing-labeled areas will be regarded as the background during training. Previous art on this problem has proposed to use soft sampling to re-weight the gradients of RoIs based on the overlaps with positive instances, while their method is mainly based on the two-stage detector (i.e. Faster RCNN) which is more robust and friendly for the missing label scenario. In this paper, we introduce a superior solution called Background Recalibration Loss (BRL) that can automatically re-calibrate the loss signals according to the pre-defined IoU threshold and input image. Our design is built on the one-stage detector which is faster and lighter. Inspired by the Focal Loss formulation, we make several significant modifications to fit on the missing-annotation circumstance. We conduct extensive experiments on the curated PASCAL VOC and MS COCO datasets. The results demonstrate that our proposed method outperforms the baseline and other state-of-the-arts by a large margin. Code available: https://github.com/Dwrety/mmdetection-selective-iou.
Politecnico di Torino（都灵理工大学）；Italian Institute of Technology（意大利技术研究院）
Despite the recent advances in the field of object detection, common architectures are still ill-suited to incrementally detect new categories over time. They are vulnerable to catastrophic forgetting: they forget what has been already learned while updating their parameters in absence of the original training data. Previous works extended standard classification methods in the object detection task, mainly adopting the knowledge distillation framework. However, we argue that object detection introduces an additional problem, which has been overlooked. While objects belonging to new classes ar…
As with other deep learning methods, label quality is important for learning modern convolutional object detectors. However, the potentially large number and wide diversity of object instances that can be found in complex image scenes makes constituting complete annotations a challenging task; objects missing annotations can be observed in a variety of popular object detection datasets. These missing annotations can be problematic, as the standard cross-entropy loss employed to train object detection models treats classification as a positive-negative (PN) problem: unlabeled regions are implic…
Recently, the availability of remote sensing imagery from aerial vehicles and satellites constantly improved. For an automated interpretation of such data, deep-learning-based object detectors achieve state-of-the-art performance. However, established object detectors require complete, precise, and correct bounding box annotations for training. In order to create the necessary training annotations for object detectors, imagery can be georeferenced and combined with data from other sources, such as points of interest localized by GPS sensors. Unfortunately, this combination often leads to poor …
Annotating bounding boxes for object detection is expensive, time-consuming, and error-prone. In this work, we propose a DETR based framework called ComplETR that is designed to explicitly complete missing annotations in partially annotated dense scene datasets. This reduces the need to annotate every object instance in the scene thereby reducing annotation cost. ComplETR augments object queries in DETR decoder with patch information of objects in the image. Combined with a matching loss, it can effectively find objects that are similar to the input patch and complete the missing annotations. …
High-resolution fingerprint recognition often relies on sophisticated matching algorithms based on hand-crafted keypoint descriptors, with pores being the most common keypoint choice. Our method is the opposite of the prevalent approach: we use instead a simple matching algorithm based on robust local pore descriptors that are learned from the data using a CNN. In order to train this CNN in a fully supervised manner, we describe how the automatic alignment of fingerprint images can be used to obtain the required training annotations, which are otherwise missing in all publicly available datase…
We study the robustness of object detection under the presence of missing annotations. In this setting, the unlabeled object instances will be treated as background, which will generate an incorrect training signal for the detector. Interestingly, we observe that after dropping 30% of the annotations (and labeling them as background), the performance of CNN-based object detectors like Faster-RCNN only drops by 5% on the PASCAL VOC dataset. We provide a detailed explanation for this result. To further bridge the performance gap, we propose a simple yet effective solution, called Soft Sampling. …
Fully supervised object detection requires training images in which all instances are annotated. This is actually impractical due to the high labor and time costs and the unavoidable missing annotations. As a result, the incomplete annotation in each image could provide misleading supervision and harm the training. Recent works on sparsely annotated object detection alleviate this problem by generating pseudo labels for the missing annotations. Such a mechanism is sensitive to the threshold of the pseudo label score. However, the effective threshold is different in different training stages an…
The application of cross-dataset training in object detection tasks is complicated because the inconsistency in the category range across datasets transforms fully supervised learning into semi-supervised learning. To address this problem, recent studies focus on the generation of high-quality missing annotations. In this study, we first point out that it is not enough to generate high-quality annotations using a single model, which only looks once for annotations. Through detailed experimental analyses, we further conclude that hard-label training is conducive to generating high-recall annota…
Columbia University（哥伦比亚大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
To extract information at scale, researchers increasingly apply semantic segmentation techniques to remotely-sensed imagery. While fully-supervised learning enables accurate pixel-wise segmentation, compiling the exhaustive datasets required is often prohibitively expensive. As a result, many non-urban settings lack the ground-truth needed for accurate segmentation. Existing open source infrastructure data for these regions can be inexact and non-exhaustive. Open source infrastructure annotations like OpenStreetMaps (OSM) are representative of this issue: while OSM labels provide global insigh…
Computer Vision Lab Delft University of Technology（代尔夫特理工大学计算机视觉实验室）；Delft University of Technology（代尔夫特理工大学）；The Netherlands（荷兰（注：原文本中The Netherlands为国家名，根据提取规则，国家不属于机构，此处仅为完整显示原机构相关信息））；Computer Vision Lab（计算机视觉实验室）
Current weakly supervised object localization and segmentation rely on class-discriminative visualization techniques to generate pseudo-labels for pixel-level training. Such visualization methods, including class activation mapping (CAM) and Grad-CAM, use only the deepest, lowest resolution convolutional layer, missing all information in intermediate layers. We propose Zoom-CAM: going beyond the last lowest resolution layer by integrating the importance maps over all activations in intermediate layers. Zoom-CAM captures fine-grained small-scale objects for various discriminative class instance…
Unsupervised Machine Learning Paradigms are often the only methodology to rely on, given a Pattern Recognition Task with no target label or annotations being present. In such scenarios, data preparation is a crucial step to be performed so that the Unsupervised Paradigms work with as much perfection as possible. But, when there is no sufficient or missing data being present in each and every instance of a dataset, data preparation becomes a challenge itself. One such case-study is the Jester Dataset that has missing values which are basically ratings given by Joke-Readers to a specified set of…
Are Dense Labels Always Necessary for 3D Object Detection from Point Cloud?
School of Intelligent Systems Engineering, the Shenzhen Campus of Sun Yatsen University（南方科技大学深圳校区智能系统工程学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Mathematics and Statistics and Ministry of Education Key Lab of Intelligent Networks and Network Security, Xi’an Jiaotong University（西安交通大学数学与统计学系和教育部智能网络与网络安全重点实验室）；School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications（重庆邮电大学通信与信息工程学院）；Meta；University of Maryland（美国马里兰大学）
DRAFE: Domain-Robust Asymmetric Fusion of Heterogeneous Detection Transformers for Cross-City Fine-Grained Traffic Object Detection
Iowa State University（爱荷华州立大学）；University of Arkansas（阿肯色大学）；Parsons Corporation（帕森斯公司）；Jacobs（雅各布斯公司）；University of South Florida（南佛罗里达大学）；Neel-Schaffer, Inc.（尼尔-沙弗公司）
针对跨城市细粒度交通目标检测中细粒度识别与域泛化难兼顾的问题，提出 DRAFE 模型，通过两阶段训练及多模块融合策略，在 AI City Challenge 2026 Track 6 任务中取得 0.4022 mAP 排名第六的成绩，性能优于初步集成模型。
Sun Yat-sen University（中山大学）；Tencent（腾讯）；Jilin University（吉林大学）；Peng Cheng Laboratory（鹏城实验室）
Object detection under imperfect data receives great attention recently. Weakly supervised object detection (WSOD) suffers from severe localization issues due to the lack of instance-level annotation, while semi-supervised object detection (SSOD) remains challenging led by the inter-image discrepancy between labeled and unlabeled data. In this study, we propose the Single Instance annotated Object Detection (SIOD), requiring only one instance annotation for each existing category in an image. Degraded from inter-task (WSOD) or inter-image (SSOD) discrepancies to the intra-image discrepancy, SI…
An important preliminary step of optical character recognition systems is the detection of text rows. To address this task in the context of historical data with missing labels, we propose a self-paced learning algorithm capable of improving the row detection performance. We conjecture that pages with more ground-truth bounding boxes are less likely to have missing annotations. Based on this hypothesis, we sort the training examples in descending order with respect to the number of ground-truth bounding boxes, and organize them into k batches. Using our self-paced learning method, we train a r…
Fully annotated large-scale medical image datasets are highly valuable. However, because labeling medical images is tedious and requires specialized knowledge, the large-scale datasets available often have missing annotation issues. For instance, DeepLesion, a large-scale CT image dataset with labels for various kinds of lesions, is reported to have a missing annotation rate of 50\%. Directly training a lesion detector on it would suffer from false negative supervision caused by unannotated lesions. To address this issue, previous works have used sophisticated multi-stage strategies to switch …
Conditional Generative Adversarial Nets (CGANs) need a significantly huge dataset with a detailed pixel-wise annotation to generate high-quality images. Unfortunately, any amount of missing pixel annotations may significantly impact the result not only locally, but also in annotated areas. To the best of our knowledge, such a challenge has never been investigated in the broader field of GANs. In this paper, we take the first step in this direction to study the problem of CGAN-based satellite image synthesis given partially annotated images. We first define the problem of image synthesis using …
European Laboratory for Non-Linear Spectroscopy (LENS), University of Florence（佛罗伦萨大学欧洲非线性光谱实验室）；Bioretics SrL（Bioretics SrL公司）；National Institute of Optics, National Research Council (INO-CNR)（国家研究委员会国家光学研究所）；Department of Physics（物理系）
Semantic segmentation of neuronal structures in 3D high-resolution fluorescence microscopy imaging of the human brain cortex can take advantage of bidimensional CNNs, which yield good results in neuron localization but lead to inaccurate surface reconstruction. 3D CNNs, on the other hand, would require manually annotated volumetric data on a large scale and hence considerable human effort. Semi-supervised alternative strategies which make use only of sparse annotations suffer from longer training times and achieved models tend to have increased capacity compared to 2D CNNs, needing more ground…
SenseTime Research（商汤科技研究院）；Dalian University of Technology（大连理工大学）
Weakly supervised object detection (WSOD) using only image-level annotations has attracted growing attention over the past few years. Existing approaches using multiple instance learning easily fall into local optima, because such mechanism tends to learn from the most discriminative object in an image for each category. Therefore, these methods suffer from missing object instances which degrade the performance of WSOD. To address this problem, this paper introduces an end-to-end object instance mining (OIM) framework for weakly supervised object detection. OIM attempts to detect all possible …
Nanyang Technological University（南洋理工大学）；Adobe Research（奥多比研究院）；University of Georgia（佐治亚大学）；Google Cloud AI（谷歌云人工智能）
Scene graph generation has received growing attention with the advancements in image understanding tasks such as object detection, attributes and relationship prediction,~\etc. However, existing datasets are biased in terms of object and relationship labels, or often come with noisy and missing annotations, which makes the development of a reliable scene graph prediction model very challenging. In this paper, we propose a novel scene graph generation algorithm with external knowledge and image reconstruction loss to overcome these dataset issues. In particular, we extract commonsense knowledge…
Zhejiang University（浙江大学）；Center for frontier AI research（前沿人工智能研究中心）
The expensive annotation cost is notoriously known as the main constraint for the development of the point cloud semantic segmentation technique. Active learning methods endeavor to reduce such cost by selecting and labeling only a subset of the point clouds, yet previous attempts ignore the spatial-structural diversity of the selected samples, inducing the model to select clustered candidates with similar shapes in a local area while missing other representative ones in the global environment. In this paper, we propose a new 3D region-based active learning method to tackle this problem. Dubbe…
Object detectors usually achieve promising results with the supervision of complete instance annotations. However, their performance is far from satisfactory with sparse instance annotations. Most existing methods for sparsely annotated object detection either re-weight the loss of hard negative samples or convert the unlabeled instances into ignored regions to reduce the interference of false negatives. We argue that these strategies are insufficient since they can at most alleviate the negative effect caused by missing annotations. In this paper, we propose a simple but effective mechanism, …
Weakly supervised instance segmentation reduces the cost of annotations required to train models. However, existing approaches which rely only on image-level class labels predominantly suffer from errors due to (a) partial segmentation of objects and (b) missing object predictions. We show that these issues can be better addressed by training with weakly labeled videos instead of images. In videos, motion and temporal consistency of predictions across frames provide complementary signals which can help segmentation. We are the first to explore the use of these video signals to tackle weakly su…
Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception
Center for Future Media and School of Computer Science and Engineering, University of Electronic Science and Technology of China（未来媒体中心和电子科技大学计算机科学与工程学院）；School of Computer Science and Technology, Tongji University（计算机科学与技术学院，同济大学）；Peng Cheng Laboratory（鹏城实验室）
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；Xomnia B.V.（Xomnia公司）；Erasmus University Medical Center（伊拉斯谟大学医学中心）；Delft University of Technology（代尔夫特理工大学）；Leiden University Medical Center（莱顿大学医学中心）
Deep learning models benefit from training with a large dataset (labeled or unlabeled). Following this motivation, we present an approach to learn a deep learning model for the automatic segmentation of Organs at Risk (OARs) in cervical cancer radiation treatment from a large clinically available dataset of Computed Tomography (CT) scans containing data inhomogeneity, label noise, and missing annotations. We employ simple heuristics for automatic data cleaning to minimize data inhomogeneity and label noise. Further, we develop a semi-supervised learning approach utilizing a teacher-student set…
Compared with multi-class classification, multi-label classification that contains more than one class is more suitable in real life scenarios. Obtaining fully labeled high-quality datasets for multi-label classification problems, however, is extremely expensive, and sometimes even infeasible, with respect to annotation efforts, especially when the label spaces are too large. This motivates the research on partial-label classification, where only a limited number of labels are annotated and the others are missing. To address this problem, we first propose a pseudo-label based approach to reduc…
PAII Inc.（PAII公司）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）；Ping An Insurance (Group) Company of China, Ltd.（中国平安保险（集团）股份有限公司）
Large-scale datasets with high-quality labels are desired for training accurate deep learning models. However, due to the annotation cost, datasets in medical imaging are often either partially-labeled or small. For example, DeepLesion is such a large-scale CT image dataset with lesions of various types, but it also has many unlabeled lesions (missing annotations). When training a lesion detector on a partially-labeled dataset, the missing annotations will generate incorrect negative signals and degrade the performance. Besides DeepLesion, there are several small single-type datasets, such as …
Temporally Consistent Mitral Annulus Measurements from Sparse Annotations in Echocardiographic Videos
Department of Biomedical Engineering & Physics, Amsterdam University Medical Center, Netherlands（生物医学工程与物理系，阿姆斯特丹大学医学中心，荷兰）；Informatics Institute, University of Amsterdam, Netherlands（信息学院，阿姆斯特丹大学，荷兰）；Amsterdam Cardiovascular Sciences, Netherlands（阿姆斯特丹心血管科学，荷兰）；Department of Biomedical Signals and Systems, University of Twente, Netherlands（生物医学信号与系统系，特文特大学，荷兰）；Department of Cardiology, Amsterdam University Medical Center, Netherlands（心脏病学系，阿姆斯特丹大学医学中心，荷兰）；Department of Radiology & Nuclear Medicine, Amsterdam University Medical Center, Netherlands（放射学与核医学系，阿姆斯特丹大学医学中心，荷兰）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Daimler TSS（戴姆勒TSS公司）；University Hospital Heidelberg（海德堡大学医院）；German Cancer Research Center（德国癌症研究中心）；University Clinic Essen（埃森大学医院）
In clinical radiology reports, doctors capture important information about the patient's health status. They convey their observations from raw medical imaging data about the inner structures of a patient. As such, formulating reports requires medical experts to possess wide-ranging knowledge about anatomical regions with their normal, healthy appearance as well as the ability to recognize abnormalities. This explicit grasp on both the patient's anatomy and their appearance is missing in current medical image-processing systems as annotations are especially difficult to gather. This renders th…
Temple University（天普大学）；Amazon Web Services（亚马逊云科技）；Stony Brook University（石溪大学）
Human annotation is always considered as ground truth in video object tracking tasks. It is used in both training and evaluation purposes. Thus, ensuring its high quality is an important task for the success of trackers and evaluations between them. In this paper, we give a qualitative and quantitative analysis of the existing human annotations. We show that human annotation tends to be non-smooth and is prone to partial visibility and deformation. We propose a smoothing trajectory strategy with the ability to handle moving scenes. We use a two-step adaptive image alignment algorithm to find t…
Head-wise Modality Specialization within MLLMs for Robust Fake News Detection under Missing Modality
Soochow University（苏州大学）；Hong Kong University of Science and Technology（香港科技大学）；Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；The Hong Kong University of Science and Technology（香港科技大学）；FiT, Tencent（腾讯FiT）；Alibaba Group（阿里巴巴集团）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Education, Zhejiang Normal University（浙江师范大学教育学院）
Tencent AI Lab（腾讯人工智能实验室）；KAUST（阿卜杜拉国王科技大学）；SUNY-Albany（纽约州立大学奥尔巴尼分校）
This work focuses on the problem of multi-label learning with missing labels (MLML), which aims to label each test instance with multiple class labels given training instances that have an incomplete/partial set of these labels. The key point to handle missing labels is propagating the label information from provided labels to missing labels, through a dependency graph that each label of each instance is treated as a node. We build this graph by utilizing different types of label dependencies. Specifically, the instance-level similarity is served as undirected edges to connect the label nodes …
Cross-head Supervision for Crowd Counting with Noisy Annotations
1 Shanghai Key Lab of Intelligent Information Processing, School of Computer Science 2 Institute of Science；Technology for Brain-inspired Intelligence Fudan University, Shanghai 200433, China
Cell detection is an essential task in cell image analysis. Recent deep learning-based detection methods have achieved very promising results. In general, these methods require exhaustively annotating the cells in an entire image. If some of the cells are not annotated (imperfect annotation), the detection performance significantly degrades due to noisy labels. This often occurs in real collaborations with biologists and even in public data-sets. Our proposed method takes a pseudo labeling approach for cell detection from imperfect annotated data. A detection convolutional neural network (CNN)…
Computer vision is driven by the many datasets available for training or evaluating novel methods. However, each dataset has a different set of class labels, visual definition of classes, images following a specific distribution, annotation protocols, etc. In this paper we explore the automatic discovery of visual-semantic relations between labels across datasets. We aim to understand how instances of a certain class in a dataset relate to the instances of another class in another dataset. Are they in an identity, parent/child, overlap relation? Or is there no link between them at all? To find…
Early diagnosis of signet ring cell carcinoma dramatically improves the survival rate of patients. Due to lack of public dataset and expert-level annotations, automatic detection on signet ring cell (SRC) has not been thoroughly investigated. In MICCAI DigestPath2019 challenge, apart from foreground (SRC region)-background (normal tissue area) class imbalance, SRCs are partially annotated due to costly medical image annotation, which introduces extra label noise. To address the issues simultaneously, we propose Decoupled Gradient Harmonizing Mechanism (DGHM) and embed it into classification lo…
Large-scale multi-label classification datasets are commonly, and perhaps inevitably, partially annotated. That is, only a small subset of labels are annotated per sample. Different methods for handling the missing labels induce different properties on the model and impact its accuracy. In this work, we analyze the partial labeling problem, then propose a solution based on two key ideas. First, un-annotated labels should be treated selectively according to two probability quantities: the class distribution in the overall dataset and the specific label likelihood for a given data sample. We pro…
