Object counting is an important task in computer vision due to its growing demand in applications such as surveillance, traffic monitoring, and counting everyday objects. State-of-the-art methods use regression-based optimization where they explicitly learn to count the objects of interest. These often perform better than detection-based methods that need to learn the more difficult task of predicting the location, size, and shape of each object. However, we propose a detection-based method that does not need to estimate the size and shape of the objects and that outperforms regression-based methods. Our contributions are three-fold: (1) we propose a novel loss function that encourages the network to output a single blob per object instance using point-level annotations only; (2) we design two methods for splitting large predicted blobs between object instances; and (3) we show that our method achieves new state-of-the-art results on several challenging datasets including the Pascal VOC and the Penguins dataset. Our method even outperforms those that use stronger supervision such as depth features, multi-point annotations, and bounding-box labels.
School of Computer Science（计算机学院）；Northwestern Polytechnical University（西北工业大学）
Crowd counting is a task worth exploring in modern society because of its wide applications such as public safety and video monitoring. Many CNN-based approaches have been proposed to improve the accuracy of estimation, but there are some inherent issues affect the performance, such as overfitting and details lost caused by pooling layers. To tackle these problems, in this paper, we propose an effective network called MDSNet, which introduces a novel supervision framework called Multi-channel Deep Supervision (MDS). The MDS conducts channel-wise supervision on the decoder of the estimation mod…
Current class-agnostic counting methods can generalise to unseen classes but usually require reference images to define the type of object to be counted, as well as instance annotations during training. Reference-less class-agnostic counting is an emerging field that identifies counting as, at its core, a repetition-recognition task. Such methods facilitate counting on a changing set composition. We show that a general feature space with global context can enumerate instances in an image without a prior on the object type present. Specifically, we demonstrate that regression from vision transf…
Inception Institute of Artificial Intelligence（安擎人工智能研究院）
Common object counting in a natural scene is a challenging problem in computer vision with numerous real-world applications. Existing image-level supervised common object counting approaches only predict the global object count and rely on additional instance-level supervision to also determine object locations. We propose an image-level supervised approach that provides both the global object count and the spatial distribution of object instances by constructing an object category density map. Motivated by psychological studies, we further reduce image-level supervision using a limited object…
University of California, Berkeley（加州大学伯克利分校）；Huazhong University of Science and Technology（华中科技大学）；Wuhan University（武汉大学）；ByteDance（字节跳动）；University of Oxford（牛津大学）
Recent works on crowd counting mainly leverage CNNs to count by regressing density maps, and have achieved great progress. In the density map, each person is represented by a Gaussian blob, and the final count is obtained from the integration of the whole map. However, it is difficult to accurately predict the density map on dense regions. A major issue is that the density map on dense regions usually accumulates density values from a number of nearby Gaussian blobs, yielding different large density values on a small set of pixels. This makes the density map present variant patterns with signi…
Cross-head Supervision for Crowd Counting with Noisy Annotations
1 Shanghai Key Lab of Intelligent Information Processing, School of Computer Science 2 Institute of Science；Technology for Brain-inspired Intelligence Fudan University, Shanghai 200433, China
Precision agriculture has become a key factor for increasing crop yields by providing essential information to decision makers. In this work, we present a deep learning method for simultaneous segmentation and counting of cranberries to aid in yield estimation and sun exposure predictions. Notably, supervision is done using low cost center point annotations. The approach, named Triple-S Network, incorporates a three-part loss with shape priors to promote better fitting to objects of known shape typical in agricultural scenes. Our results improve overall segmentation performance by more than 6.…
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；ETH Zurich（苏黎世联邦理工学院）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Generic object counting in natural scenes is a challenging computer vision problem. Existing approaches either rely on instance-level supervision or absolute count information to train a generic object counter. We introduce a partially supervised setting that significantly reduces the supervision level required for generic object counting. We propose two novel frameworks, named lower-count (LC) and reduced lower-count (RLC), to enable object counting under this setting. Our frameworks are built on a novel dual-branch architecture that has an image classification and a density branch. Our LC fr…
Instituto de Física de São Carlos - Universidade de São Paulo（圣保罗大学圣卡洛斯物理研究所）
This paper presents a new method for automatic quantification of ellipse-like cells in images, an important and challenging problem that has been studied by the computer vision community. The proposed method can be described by two main steps. Initially, image segmentation based on the k-means algorithm is performed to separate different types of cells from the background. Then, a robust and efficient strategy is performed on the blob contour for touching cells splitting. Due to the contour processing, the method achieves excellent results of detection compared to manual detection performed by…
To learn a reliable people counter from crowd images, head center annotations are normally required. Annotating head centers is however a laborious and tedious process in dense crowds. In this paper, we present an active learning framework which enables accurate crowd counting with limited supervision: given a small labeling budget, instead of randomly selecting images to annotate, we first introduce an active labeling strategy to annotate the most informative images in the dataset and learn the counting model upon them. The process is repeated such that in every cycle we select the samples th…
Counting Trees from Satellite Imagery with Noisy Supervision
University of Copenhagen（哥本哈根大学）；University of Rwanda（卢旺达大学）；University of Chinese Academy of Sciences（中国科学院大学）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Stony Brook University（石溪大学）；LIGM, CNRS, Univ Gustave Eiffel, ENPC, IPP（LIGM，CNRS，古斯塔夫·埃菲尔大学，ENPC，IPP）
Crowd counting aims to learn the crowd density distributions and estimate the number of objects (e.g. persons) in images. The perspective effect, which significantly influences the distribution of data points, plays an important role in crowd counting. In this paper, we propose a novel perspective-aware approach called PANet to address the perspective problem. Based on the observation that the size of the objects varies greatly in one image due to the perspective effect, we propose the dynamic receptive fields (DRF) framework. The framework is able to adjust the receptive field by the dilated …
Stony Brook University（石溪大学）；University of Oxford（牛津大学）
Crowd-sourcing is an increasingly popular tool for image analysis in animal ecology. Computer vision methods that can utilize crowd-sourced annotations can help scale up analysis further. In this work we study the potential to do so on the challenging task of fine-grained counting. As opposed to the standard crowd counting task, fine-grained counting also involves classifying attributes of individuals in dense crowds. We introduce a new dataset from animal ecology to enable this study that contains 1.7M crowd-sourced annotations of 8 fine-grained classes. It is the largest available dataset fo…
Ingenuity Labs Research Institute, Queen's University（女王大学智慧实验室研究所）
We propose a multitask approach for crowd counting and person localization in a unified framework. As the detection and localization tasks are well-correlated and can be jointly tackled, our model benefits from a multitask solution by learning multiscale representations of encoded crowd images, and subsequently fusing them. In contrast to the relatively more popular density-based methods, our model uses point supervision to allow for crowd locations to be accurately identified. We test our model on two popular crowd counting datasets, ShanghaiTech A and B, and demonstrate that our method achie…
Acquiring count annotations generally requires less human effort than point-level and bounding box annotations. Thus, we propose the novel problem setup of localizing objects in dense scenes under this weaker supervision. We propose LOOC, a method to Localize Overlapping Objects with Count supervision. We train LOOC by alternating between two stages. In the first stage, LOOC learns to generate pseudo point-level annotations in a semi-supervised manner. In the second stage, LOOC uses a fully-supervised localization method that trains on these pseudo labels. The localization method is used to pr…
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density. While effective, these data-driven approaches rely on large amount of data annotation to achieve good performance, which stops these models from being deployed in emergencies during which data annotation is either too costly or cannot be obtained fast enough. One popular solution is to use synthetic data for training. Unfortunately, due to domain shift, the resulting models generalize poorly on real imagery. We remedy this shortcoming by training with both synthetic images, along wit…
Indian Institute of Science（印度科学理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Dense crowd counting is a challenging task that demands millions of head annotations for training models. Though existing self-supervised approaches could learn good representations, they require some labeled data to map these features to the end task of density estimation. We mitigate this issue with the proposed paradigm of complete self-supervision, which does not need even a single labeled image. The only input required to train, apart from a large set of unlabeled crowd images, is the approximate upper limit of the crowd count for the given dataset. Our method dwells on the idea that natu…
College of Electronics and Information Engineering, Sichuan University（四川大学电子信息学院）；School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Computer Science, The University of Adelaide（阿德莱德大学计算机学院）
Most existing crowd counting methods require object location-level annotation, i.e., placing a dot at the center of an object. While being simpler than the bounding-box or pixel-level annotation, obtaining this annotation is still labor-intensive and time-consuming especially for images with highly crowded scenes. On the other hand, weaker annotations that only know the total count of objects can be almost effortless in many practical scenarios. Thus, it is desirable to develop a learning method that can effectively train models from count-level annotations. To this end, this paper studies the…
The mainstream crowd counting methods regress density map and integrate it to obtain counting results. Since the density representation to one head accords to its adjacent distribution, it embeds the same category objects with variant values, while human beings counting models the invariant features namely similarity to objects. Inspired by this, we propose a rational and anthropoid crowd counting framework. To begin with, we leverage counting scalar as supervision signal, which provides global and implicit guidance to similar matters. Then, the large kernel CNN is utilized to imitate the para…
Accurately counting cells in microscopic images is important for medical diagnoses and biological studies, but manual cell counting is very tedious, time-consuming, and prone to subjective errors, and automatic counting can be less accurate than desired. To improve the accuracy of automatic cell counting, we propose here a novel method that employs deeply-supervised density regression. A fully convolutional neural network (FCNN) serves as the primary FCNN for density map regression. Innovatively, a set of auxiliary FCNNs are employed to provide additional supervision for learning the intermedi…
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University（西安交通大学电子与信息工程学院）
In crowd counting datasets, each person is annotated by a point, which is usually the center of the head. And the task is to estimate the total count in a crowd scene. Most of the state-of-the-art methods are based on density map estimation, which convert the sparse point annotations into a "ground truth" density map through a Gaussian kernel, and then use it as the learning target to train a density map estimator. However, such a "ground-truth" density map is imperfect due to occlusions, perspective effects, variations in object shapes, etc. On the contrary, we propose \emph{Bayesian loss}, a…
Object counting, which aims to count the accurate number of object instances in images, has been attracting more and more attention. However, challenges such as large scale variation, complex background interference, and non-uniform density distribution greatly limit the counting accuracy, particularly striking in remote sensing imagery. To mitigate the above issues, this paper proposes a novel framework for dense object counting in remote sensing images, which incorporates a pyramidal scale module (PSM) and a global context module (GCM), dubbed PSGCNet, where PSM is used to adaptively capture…
We tackle a new task of few-shot object counting and detection. Given a few exemplar bounding boxes of a target object class, we seek to count and detect all objects of the target class. This task shares the same supervision as the few-shot object counting but additionally outputs the object bounding boxes along with the total object count. To address this challenging problem, we introduce a novel two-stage training strategy and a novel uncertainty-aware few-shot object detector: Counting-DETR. The former is aimed at generating pseudo ground-truth bounding boxes to train the latter. The latter…
Object Counting with GPT-4o and GPT-5: A Comparative Study
Software Engineering Institute, Obuda University, Budapest, Hungary（奥布达大学软件工程研究所）；Doctoral School of Applied Informatics（应用信息科学博士学院）；Applied Mathematics, Obuda University, Budapest, Hungary（应用数学，奥布达大学）；John von Neumann Faculty of Informatics, Obuda University, Budapest, Hungary（约翰·冯·诺伊曼信息学院，奥布达大学）；Faculty of Economics（经济学院）
University of Bern（伯尔尼大学）；University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）
We introduce a novel method for representation learning that uses an artificial supervision signal based on counting visual primitives. This supervision signal is obtained from an equivariance relation, which does not require any manual annotation. We relate transformations of images to transformations of the representations. More specifically, we look for the representation that satisfies such relation rather than the transformations that match a given representation. In this paper, we use two image transformations in the context of counting: scaling and tiling. The first transformation explo…
This paper aims to count arbitrary objects in images. The leading counting approaches start from point annotations per object from which they construct density maps. Then, their training objective transforms input images to density maps through deep convolutional networks. We posit that the point annotations serve more supervision purposes than just constructing density maps. We introduce ways to repurpose the points for free. First, we propose supervised focus from segmentation, where points are converted into binary maps. The binary maps are combined with a network branch and accompanying lo…
Universitat de Barcelona（巴塞罗那大学）；Universidade Federal de Santa Catarina（圣卡塔琳娜联邦大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Universidade Federal do Paraná（巴拉那联邦大学）；Witwatersrand University（威特沃特斯兰德大学）；Oregon State University（俄勒冈州立大学）；University of Zurich（苏黎世大学）；Universitat Autonoma de Barcelona（巴塞罗那自治大学）
Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the …
University of Liverpool（利物浦大学）；Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Science（中国科学院宁波材料技术与工程研究所慈溪生物医学工程研究所）；China Science IntelliCloud Technology Co., Ltd（中国科聪智能科技有限公司）；Remark AI UK Limited（英国锐马可人工智能有限公司）
This paper proposes an adaptive auxiliary task learning based approach for object counting problems. Unlike existing auxiliary task learning based methods, we develop an attention-enhanced adaptively shared backbone network to enable both task-shared and task-tailored features learning in an end-to-end manner. The network seamlessly combines standard Convolution Neural Network (CNN) and Graph Convolution Network (GCN) for feature extraction and feature reasoning among different domains of tasks. Our approach gains enriched contextual information by iteratively and hierarchically fusing the fea…
College of Computer Science, Sichuan University（四川大学计算机学院）；Univ Rennes, Inria, CNRS, IRISA（雷恩大学、法国国家信息与自动化研究所、法国国家科学研究中心、雷恩第一大学信息与应用数学研究所）
Modern crowd counting methods usually employ deep neural networks (DNN) to estimate crowd counts via density regression. Despite their significant improvements, the regression-based methods are incapable of providing the detection of individuals in crowds. The detection-based methods, on the other hand, have not been largely explored in recent trends of crowd counting due to the needs for expensive bounding box annotations. In this work, we instead propose a new deep detection network with only point supervision required. It can simultaneously detect the size and location of human heads and co…
Shanghai Jiao Tong University（上海交通大学）；University of Technology, Sydney（悉尼科技大学）
High-density object counting in surveillance scenes is challenging mainly due to the drastic variation of object scales. The prevalence of deep learning has largely boosted the object counting accuracy on several benchmark datasets. However, does the global counts really count? Armed with this question we dive into the predicted density map whose summation over the whole regions reports the global counts for more in-depth analysis. We observe that the object density map generated by most existing methods usually lacks of local consistency, i.e., counting errors in local regions exist unexpecte…
T2ICount: Enhancing Cross-modal Understanding for Zero-Shot Counting
University of Nottingham（诺丁汉大学）；University of St Andrews（圣安德鲁大学）；City University of Hong Kong（香港城市大学）；Nanyang Technology University（南洋理工大学）；Harbin Institute of Technology（哈尔滨工业大学）
Leveraging Weak Supervision for Cell Localization in Digital Pathology Using Multitask Learning and Consistency Loss
Koc University（科克大学）；Basaksehir Cam and Sakura City Hospital（巴沙克谢伊尔卡姆与樱花城市医院）；Sisli Hamidiye Etfal Health Application and Research Center（锡什利哈米迪耶埃特法尔健康应用与研究中心）；Koc University School of Medicine（科克大学医学院）
Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting
College of Electronic and Information Engineering, Tongji University（电子信息工程学院，同济大学）；College of Computer Science, Wuhan University（计算机科学学院，武汉大学）；College of Computer Science, Tongji University（计算机科学学院，同济大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）
Inception Institute of Artificial Intelligence（Inception人工智能研究院）
Temporal action localization is a challenging computer vision problem with numerous real-world applications. Most existing methods require laborious frame-level supervision to train action localization models. In this work, we propose a framework, called 3C-Net, which only requires video-level supervision (weak supervision) in the form of action category labels and the corresponding count. We introduce a novel formulation to learn discriminative action features with enhanced localization capabilities. Our joint formulation has three terms: a classification term to ensure the separability of le…
Tsinghua University（清华大学）；China University of Geosciences, Wuhan（武汉地质大学）；State Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）；National Engineering Research Center for Visual Information and Applications（视觉信息与应用国家工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）
A unified three-branch deep learning framework for automated detection and counting of wildlife in aerial imagery
School of Geospatial Engineering and Science, Sun Yat-Sen University（中山大学地理空间工程与科学学院）；Faculty of Geo-Information Science and Earth Observation, University of Twente（特温特大学地球信息科学与对地观测学院）；Boma and Badingilo National Parks, African Parks（博马和巴丁吉洛国家公园，非洲公园组织）；Department of Ecology and Evolutionary Biology, Princeton University（普林斯顿大学生态与进化生物学系）；Lolldaiga Hills Research Programme（洛尔迪加山研究项目）；School of Biological, Earth & Environmental Sciences, Environmental Research Institute, University College Cork（科克大学环境研究所生物、地球与环境科学学院）
CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of EIC, Huazhong University of Science and Technology（华中科技大学电子信息与工程学院）；Tencent TEG（腾讯技术工程事业群）
