We introduce count-guided weakly supervised localization (C-WSL), an approach that uses per-class object count as a new form of supervision to improve weakly supervised localization (WSL). C-WSL uses a simple count-based region selection algorithm to select high-quality regions, each of which covers a single object instance during training, and improves existing WSL methods by training with the selected regions. To demonstrate the effectiveness of C-WSL, we integrate it into two WSL architectures and conduct extensive experiments on VOC2007 and VOC2012. Experimental results show that C-WSL leads to large improvements in WSL and that the proposed approach significantly outperforms the state-of-the-art methods. The results of annotation experiments on VOC2007 suggest that a modest extra time is needed to obtain per-class object counts compared to labeling only object categories in an image. Furthermore, we reduce the annotation time by more than $2\times$ and $38\times$ compared to center-click and bounding-box annotations.
University of Maryland（马里兰大学）；Salesforce Research（Salesforce研究院）
We propose weakly supervised language localization networks (WSLLN) to detect events in long, untrimmed videos given language queries. To learn the correspondence between visual segments and texts, most previous methods require temporal coordinates (start and end times) of events for training, which leads to high costs of annotation. WSLLN relieves the annotation burden by training with only video-sentence pairs without accessing to temporal locations of events. With a simple end-to-end structure, WSLLN measures segment-text consistency and conducts segment selection (conditioned on the text) …
École de technologie supérieure（高等技术学院）；Goodman Cancer Research Centre（古德曼癌症研究中心）；McGill University（麦吉尔大学）
Weakly-supervised learning (WSL) has recently triggered substantial interest as it mitigates the lack of pixel-wise annotations. Given global image labels, WSL methods yield pixel-level predictions (segmentations), which enable to interpret class predictions. Despite their recent success, mostly with natural images, such methods can face important challenges when the foreground and background regions have similar visual cues, yielding high false-positive rates in segmentations, as is the case in challenging histology images. WSL training is commonly driven by standard classification losses, wh…
École de technologie supérieure, Université du Québec（魁北克大学高等技术学院）；Rosalind and Morris Goodman Cancer Research Centre（罗莎琳德与莫里斯·古德曼癌症研究中心）；McGill University（麦吉尔大学）
Pointwise localization allows more precise localization and accurate interpretability, compared to bounding box, in applications where objects are highly unstructured such as in medical domain. In this work, we focus on weakly supervised localization (WSL) where a model is trained to classify an image and localize regions of interest at pixel-level using only global image annotation. Typical convolutional attentions maps are prune to high false positive regions. To alleviate this issue, we propose a new deep learning method for WSL, composed of a localizer and a classifier, where the localizer…
Istituto Italiano di Tecnologia（意大利技术研究院）；Universitá degli Studi di Genova（热那亚大学）
Deep Learning (DL) based methods for object detection achieve remarkable performance at the cost of computationally expensive training and extensive data labeling. Robots embodiment can be exploited to mitigate this burden by acquiring automatically annotated training data via a natural interaction with a human showing the object of interest, handheld. However, learning solely from this data may introduce biases (the so-called domain shift), and prevents adaptation to novel tasks. While Weakly-supervised Learning (WSL) offers a well-established set of techniques to cope with these problems in …
Beihang University（北京航空航天大学）；The University of Hong Kong（香港大学）；Deepwise AI Lab（深睿人工智能实验室）；Hwa Mei Hospital, UCAS（中国科学院大学华美医院）；Ningbo Institute of Life and Health Industry, UCAS（中国科学院大学宁波生命与健康产业研究院）；Xidian University（西安电子科技大学）；Northwestern Polytechnical University（西北工业大学）
Although deep learning algorithms have been intensively developed for computer-aided tuberculosis diagnosis (CTD), they mainly depend on carefully annotated datasets, leading to much time and resource consumption. Weakly supervised learning (WSL), which leverages coarse-grained labels to accomplish fine-grained tasks, has the potential to solve this problem. In this paper, we first propose a new large-scale tuberculosis (TB) chest X-ray dataset, namely the tuberculosis chest X-ray attribute dataset (TBX-Att), and then establish an attribute-assisted weakly-supervised framework to classify and …
Food classification from images is a fine-grained classification problem. Manual curation of food images is cost, time and scalability prohibitive. On the other hand, web data is available freely but contains noise. In this paper, we address the problem of classifying food images with minimal data curation. We also tackle a key problems with food images from the web where they often have multiple cooccuring food types but are weakly labeled with a single label. We first demonstrate that by sequentially adding a few manually curated samples to a larger uncurated dataset from two web sources, th…
Dense anticipation aims to forecast future actions and their durations for long horizons. Existing approaches rely on fully-labelled data, i.e. sequences labelled with all future actions and their durations. We present a (semi-) weakly supervised method using only a small number of fully-labelled sequences and predominantly sequences in which only the (one) upcoming action is labelled. To this end, we propose a framework that generates pseudo-labels for future actions and their durations and adaptively refines them through a refinement module. Given only the upcoming action label as input, the…
In this paper, we address the problem of weakly supervised object localization (WSL), which trains a detection network on the dataset with only image-level annotations. The proposed approach is built on the observation that the proposal set from the training dataset is a collection of background, object parts, and objects. Several strategies are taken to adaptively eliminate the noisy proposals and generate pseudo object-level annotations for the weakly labeled dataset. A multiple instance learning (MIL) algorithm enhanced by mask-out strategy is adopted to collect the class-specific object pr…
SenseTime Research（商汤科技研究院）；The Chinese University of Hong Kong（香港中文大学）
The performance of object detection, to a great extent, depends on the availability of large annotated datasets. To alleviate the annotation cost, the research community has explored a number of ways to exploit unlabeled or weakly labeled data. However, such efforts have met with limited success so far. In this work, we revisit the problem with a pragmatic standpoint, trying to explore a new balance between detection performance and annotation cost by jointly exploiting fully and weakly annotated data. Specifically, we propose a weakly- and semi-supervised object detection framework (WSSOD), w…
Peripancreatic vessel segmentation and anatomical labeling play extremely important roles to assist the early diagnosis, surgery planning and prognosis for patients with pancreatic tumors. However, most current techniques cannot achieve satisfactory segmentation performance for peripancreatic veins and usually make predictions with poor integrity and connectivity. Besides, unsupervised labeling algorithms cannot deal with complex anatomical variation while fully supervised methods require a large number of voxel-wise annotations for training, which is very labor-intensive and time-consuming. T…
National University of Singapore（新加坡国立大学）；Tencent AI Lab（腾讯AI实验室）
Most existing weakly supervised localization (WSL) approaches learn detectors by finding positive bounding boxes based on features learned with image-level supervision. However, those features do not contain spatial location related information and usually provide poor-quality positive samples for training a detector. To overcome this issue, we propose a deep self-taught learning approach, which makes the detector learn the object-level features reliable for acquiring tight positive samples and afterwards re-train itself based on them. Consequently, the detector progressively improves its dete…
Arizona State University（亚利桑那州立大学）；Barrow Neurological Institute（巴罗神经学研究所）；Irkutsk State Medical University（伊尔库茨克州立医科大学）
Confocal Laser Endomicroscope (CLE) is a novel handheld fluorescence imaging device that has shown promise for rapid intraoperative diagnosis of brain tumor tissue. Currently CLE is capable of image display only and lacks an automatic system to aid the surgeon in analyzing the images. The goal of this project was to develop a computer-aided diagnostic approach for CLE imaging of human glioma with feature localization function. Despite the tremendous progress in object detection and image segmentation methods in recent years, most of such methods require large annotated datasets for training. H…
MambaEviScrib: Mamba and Evidence-Guided Consistency Enhance CNN Robustness for Scribble-Based Weakly Supervised Ultrasound Image Segmentation
School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；The SMART (Smart Medicine and AI-based Radiology Technology) Lab, Shanghai Institute for Advanced Communication and Data Science, Shanghai University（上海大学先进通信与数据科学研究院智能医学与AI放射技术实验室）；School of Health Science and Engineering, University of Shanghai for Science and Technology（上海科技大学健康科学与工程学院）；Department of Pediatric Cardiology, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院儿童心脏科部）；School of Medical Instruments, Shanghai University of Medicine and Health Sciences（医学仪器学院，上海医学院与健康科学大学）
Huazhong University of Science and Technology（华中科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
The mainstream crowd counting methods usually utilize the convolution neural network (CNN) to regress a density map, requiring point-level annotations. However, annotating each person with a point is an expensive and laborious process. During the testing phase, the point-level annotations are not considered to evaluate the counting accuracy, which means the point-level annotations are redundant. Hence, it is desirable to develop weakly-supervised counting methods that just rely on count-level annotations, a more economical way of labeling. Current weakly-supervised counting methods adopt the C…
YoloCurvSeg: You Only Label One Noisy Skeleton for Vessel-style Curvilinear Structure Segmentation
Southern University of Science and Technology（南方科技大学）；the University of Hong Kong（香港大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Fully-supervised crowd counting is a laborious task due to the large amounts of annotations. Few works focus on weekly-supervised crowd counting, where only the global crowd numbers are available for training. The main challenge of weekly-supervised crowd counting is the lack of local supervision information. To address this problem, we propose a self-adaptive feature similarity learning (SFSL) network and a global-local consistency (GLC) loss to reinforce local feature representation. We introduce a feature vector which represents the unbiased feature estimation of persons. The network update…
This paper reports a new solution of leveraging temporal classification to support weakly supervised object detection (WSOD). Specifically, we introduce raster scan-order techniques to serialize 2D images into 1D sequence data, and then leverage a combined LSTM (Long, Short-Term Memory) and CTC (Connectionist Temporal Classification) network to achieve object localization based on a total count (of interested objects). We term our proposed network LSTM-CCTC (Count-based CTC). This "learning from counting" strategy differs from existing WSOD methods in that our approach automatically identifies…
Chongqing University（重庆大学）；Huazhong University of Science and Technology（华中科技大学）
Currently, for crowd counting, the fully supervised methods via density map estimation are the mainstream research directions. However, such methods need location-level annotation of persons in an image, which is time-consuming and laborious. Therefore, the weakly supervised method just relying upon the count-level annotation is urgently needed. Since CNN is not suitable for modeling the global context and the interactions between image patches, crowd counting with weakly supervised learning via CNN generally can not show good performance. The weakly supervised model via Transformer was sequen…
Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting
University of Chinese Academy of Science（中国科学院大学）；Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；University of California, Merced（加州大学默塞德分校）；Peng Cheng Laboratory（鹏城实验室）
Bronchovascular Tree-Guided Weakly Supervised Learning Method for Pulmonary Segment Segmentation
Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院放射科）；Canon Medical Systems (China)（佳能医疗系统（中国））
Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting
College of Electronic and Information Engineering, Tongji University（电子信息工程学院，同济大学）；College of Computer Science, Wuhan University（计算机科学学院，武汉大学）；College of Computer Science, Tongji University（计算机科学学院，同济大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）
Inception Institute of Artificial Intelligence（Inception人工智能研究院）
Temporal action localization is a challenging computer vision problem with numerous real-world applications. Most existing methods require laborious frame-level supervision to train action localization models. In this work, we propose a framework, called 3C-Net, which only requires video-level supervision (weak supervision) in the form of action category labels and the corresponding count. We introduce a novel formulation to learn discriminative action features with enhanced localization capabilities. Our joint formulation has three terms: a classification term to ensure the separability of le…
Weakly supervised framework for wildlife detection and counting in challenging Arctic environments: a case study on caribou (Rangifer tarandus)
Department of Applied Geomatics, Université de Sherbrooke（应用测绘系，谢伯罗克大学）；Quebec Centre for Biodiversity Science (QCBS), Stewart Biology, McGill University（魁北克生物多样性科学中心（QCBS）、斯图尔特生物学、麦吉尔大学）
C2RM-Seg: Causal Counterfactual Reasoning with Structural-Semantic Priors for Weakly Supervised Histopathological Tissue Segmentation
Guangxi Key Laboratory of Image and Graphic Intelligent Processing, Guilin University of Electronic Technology（广西图像图形智能处理重点实验室，桂林电子科技大学）；International Joint Research Laboratory of Spatio-temporal Information and Intelligent Location Services, Guilin University of Electronic Technology（时空信息与智能定位服务国际联合研究实验室，桂林电子科技大学）；School of Computer Science and Information Security, Guilin University of Electronic Technology（桂林电子科技大学计算机与信息安全学院）
South China University of Technology（华南理工大学）；Fuzhou University（福州大学）
Crowd image is arguably one of the most laborious data to annotate. In this paper, we devote to reduce the massive demand of densely labeled crowd data, and propose a novel weakly-supervised setting, in which we leverage the binary ranking of two images with high-contrast crowd counts as training guidance. To enable training under this new setting, we convert the crowd count regression problem to a ranking potential prediction problem. In particular, we tailor a Siamese Ranking Network that predicts the potential scores of two images indicating the ordering of the counts. Hence, the ultimate g…
University of Guelph（圭尔夫大学）；Memorial University of Newfoundland（纽芬兰纪念大学）；Dalian Maritime University（大连海事大学）
Existing state-of-the-art crowd counting algorithms rely excessively on location-level annotations, which are burdensome to acquire. When only count-level (weak) supervisory signals are available, it is arduous and error-prone to regress total counts due to the lack of explicit spatial constraints. To address this issue, a novel and efficient counter (referred to as CrowdMLP) is presented, which probes into modelling global dependencies of embeddings and regressing total counts by devising a multi-granularity MLP regressor. In specific, a locally-focused pre-trained frontend is cascaded to ext…
Accurate segmentation of organelle instances, e.g., mitochondria, is essential for electron microscopy analysis. Despite the outstanding performance of fully supervised methods, they highly rely on sufficient per-pixel annotated data and are sensitive to domain shift. Aiming to develop a highly annotation-efficient approach with competitive performance, we focus on weakly-supervised domain adaptation (WDA) with a type of extremely sparse and weak annotation demanding minimal annotation efforts, i.e., sparse point annotations on only a small subset of object instances. To reduce performance deg…
Arizona State University（亚利桑那州立大学）；University of California, Irvine（加州大学欧文分校）
Computer Vision applications often require a textual grounding module with precision, interpretability, and resilience to counterfactual inputs/queries. To achieve high grounding precision, current textual grounding methods heavily rely on large-scale training data with manual annotations at the pixel level. Such annotations are expensive to obtain and thus severely narrow the model's scope of real-world applications. Moreover, most of these methods sacrifice interpretability, generalizability, and they neglect the importance of being resilient to counterfactual inputs. To address these issues…
Model pre-training is a cornerstone of modern visual recognition systems. Although fully supervised pre-training on datasets like ImageNet is still the de-facto standard, recent studies suggest that large-scale weakly supervised pre-training can outperform fully supervised approaches. This paper revisits weakly-supervised pre-training of models using hashtag supervision with modern versions of residual networks and the largest-ever dataset of images and corresponding hashtags. We study the performance of the resulting models in various transfer-learning settings including zero-shot transfer. W…
MVTec Software GmbH（MVTec软件有限公司）；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））
Grocery stores have thousands of products that are usually identified using barcodes with a human in the loop. For automated checkout systems, it is necessary to count and classify the groceries efficiently and robustly. One possibility is to use a deep learning algorithm for instance-aware semantic segmentation. Such methods achieve high accuracies but require a large amount of annotated training data. We propose a system to generate the training annotations in a weakly supervised manner, drastically reducing the labeling effort. We assume that for each training image, only the object class i…
Faculty of Science（理学院）；University of Ontario Institute of Technology（安大略理工大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；National Research Council Canada（加拿大国家研究委员会）
We demonstrate the use of an extensive deep neural network to localize instances of objects in images. The EDNN is naturally able to accurately perform multi-class counting using only ground truth count values as labels. Without providing any conceptual information, object annotations, or pixel segmentation information, the neural network is able to formulate its own conceptual representation of the items in the image. Using images labelled with only the counts of the objects present,the structure of the extensive deep neural network can be exploited to perform localization of the objects with…
National University of Singapore（新加坡国立大学）；A*STAR（新加坡科技研究局）；A*STAR Genome Institute of Singapore（新加坡科技研究局基因组研究所）；Bioinformatics Institute（生物信息学研究所）；Image Pervasive Access Lab (IPAL) CNRS UMI 2955（图像普适访问实验室（IPAL）法国国家科学研究中心联合研究单位2955）；Singapore Eye Research Institute（新加坡眼科研究所）
A weakly supervised learning based clustering framework is proposed in this paper. As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count ($ucc$), which is the number of unique classes among all instances inside the bag. In this task, no annotations on individual instances inside the bag are needed during training of the models. We mathematically prove that with a perfect $ucc$ classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are g…
TCFormer: A 5M-Parameter Transformer with Density-Guided Aggregation for Weakly-Supervised Crowd Counting
College of Mechanical and Electronic Engineering, Dalian Minzu University（大连民族大学机械与电子工程学院）；Dalian University of Technology（大连理工大学）；Postdoctoral workstation of Dalian Rijia Electronics Co., Ltd.（大连瑞嘉电子有限公司博士后工作站）；School of Computer Science and Engineering, Dalian Minzu University（大连民族大学计算机科学与工程学院）；R&D Department of Dalian Rijia Electronics Co., Ltd.（大连瑞嘉电子有限公司研发部）
College of Electronics and Information Engineering, Sichuan University（四川大学电子信息学院）；School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Computer Science, The University of Adelaide（阿德莱德大学计算机学院）
Most existing crowd counting methods require object location-level annotation, i.e., placing a dot at the center of an object. While being simpler than the bounding-box or pixel-level annotation, obtaining this annotation is still labor-intensive and time-consuming especially for images with highly crowded scenes. On the other hand, weaker annotations that only know the total count of objects can be almost effortless in many practical scenarios. Thus, it is desirable to develop a learning method that can effectively train models from count-level annotations. To this end, this paper studies the…
Two major causes of death in the United States and worldwide are stroke and myocardial infarction. The underlying cause of both is thrombi released from ruptured or eroded unstable atherosclerotic plaques that occlude vessels in the heart (myocardial infarction) or the brain (stroke). Clinical studies show that plaque composition plays a more important role than lesion size in plaque rupture or erosion events. To determine the plaque composition, various cell types in 3D cardiovascular immunofluorescent images of plaque lesions are counted. However, counting these cells manually is expensive, …
IgCONDA-PET: Weakly-Supervised PET Anomaly Detection using Implicitly-Guided Attention-Conditional Counterfactual Diffusion Modeling -- a Multi-Center, Multi-Cancer, and Multi-Tracer Study
organization= Department of Physics \& Astronomy, University of British Columbia , city= Vancouver , state= BC , country= Canada；organization= Department of Integrative Oncology, BC Cancer Research Institute , city= Vancouver , state= BC , country= Canada；organization= Department of Radiology, University of British Columbia , city= Vancouver , state= BC , country= Canada
