Counting is a fundamental operation for various real-world visual tasks, requiring both object recognition and robust counting capabilities. Despite their advanced visual perception, large vision-language models (LVLMs) are known to struggle with counting tasks. In this work, we evaluate the performance of several LVLMs on visual counting tasks across multiple counting and vision datasets. We observe that while their performance may be less prone to error for small numbers of objects, they exhibit significant weaknesses as the number of objects increases. To alleviate this issue, we propose a simple yet effective baseline method that enhances LVLMs' counting ability for large numbers of objects using a divide-and-conquer approach. Our method decomposes counting problems into sub-tasks. Moreover, it incorporates a mechanism to prevent objects from being split during division, which could otherwise lead to repetitive counting -- a common issue in a naive divide-and-conquer implementation. We demonstrate the effectiveness of this approach across various datasets and benchmarks, establishing it as a valuable reference for evaluating future solutions.
GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing
Beihang University（北京航空航天大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
Detecting Clinical Hallucinations in LVLMs via Counterfactual Visual Grounding Uncertainty
School of Intelligent Science and Technology, Nanjing University（南京大学智能科学与技术学院）；National Institute of Healthcare Data Science at Nanjing University（南京大学医疗数据科学国家研究院）；School of Biomedical Engineering, Nanjing University（南京大学生物医学工程学院）
SHIELD: Suppressing Hallucinations In LVLM Encoders via Bias and Vulnerability Defense
Department of Electrical and Computer Engineering, Northeastern University（电气与计算机工程系，东北大学）；Khoury College of Computer Science, Northeastern University（计算机科学学院，东北大学）
Self-improvement has emerged as a mainstream paradigm for advancing the reasoning capabilities of large vision-language models (LVLMs), where models explore and learn from successful trajectories iteratively. However, we identify a critical issue during this process: the model excels at generating high-quality trajectories for simple queries (i.e., head data) but struggles with more complex ones (i.e., tail data). This leads to an imbalanced optimization that drives the model to prioritize simple reasoning skills, while hindering its ability to tackle more complex reasoning tasks. Over iterati…
Fighting Fire with Fire (F3): A Training-free and Efficient Visual Adversarial Example Purification Method in LVLMs
Tsinghua University, Tencent（清华大学，腾讯）；Tencent（腾讯）；University of Science and Technology Beijing（北京科技大学）；Tencent, University of Macau（腾讯，澳门大学）；Tsinghua University（清华大学）
Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Nanjing University（南京大学）；Tongji University（同济大学）；Wuhan University（武汉大学）
HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology（香港科技大学）；Huawei Research（华为研究）；Beihang University（北航）
Effectiveness Assessment of Recent Large Vision-Language Models
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Sichuan University（四川大学）；Tianjin University（天津大学）；Australian National University（澳大利亚国立大学）；Harbin Institute of Technology（哈尔滨工业大学）；Nankai University（南开大学）
Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference Framework
Tongji University（同济大学）；Computer Network Information Center, CAS（计算机网络信息中心，中国科学院）；HIAS, University of Chinese Academy of Sciences（高等研究所，中国科学院大学）；University of Chinese Academy of Sciences（中国科学院大学）
Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
Hong Kong University of Science and Technology（香港科技大学）；Dalian University of Technology（大连理工大学）；The Chinese University of Hong Kong（香港中文大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Virginia Tech（弗吉尼亚理工大学）；Georgia Institute of Technology（佐治亚理工学院）
We are interested in counting the number of instances of object classes in natural, everyday images. Previous counting approaches tackle the problem in restricted domains such as counting pedestrians in surveillance videos. Counts can also be estimated from outputs of other vision tasks like object detection. In this work, we build dedicated models for counting designed to tackle the large variance in counts, appearances, and scales of objects found in natural scenes. Our approach is inspired by the phenomenon of subitizing - the ability of humans to make quick assessments of counts given a pe…
The mainstream crowd counting methods regress density map and integrate it to obtain counting results. Since the density representation to one head accords to its adjacent distribution, it embeds the same category objects with variant values, while human beings counting models the invariant features namely similarity to objects. Inspired by this, we propose a rational and anthropoid crowd counting framework. To begin with, we leverage counting scalar as supervision signal, which provides global and implicit guidance to similar matters. Then, the large kernel CNN is utilized to imitate the para…
Counting Beyond Instances: A Benchmark for Group-Individual Object Counting
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Liverpool（利物浦大学）
Beijing University of Posts and Telecommunications（北京邮电大学）；Huazhong University of Science and Technology（华中科技大学）
Object counting aims to estimate the number of objects in images. The leading counting approaches focus on the single category counting task and achieve impressive performance. Note that there are multiple categories of objects in real scenes. Multi-class object counting expands the scope of application of object counting task. The multi-target detection task can achieve multi-class object counting in some scenarios. However, it requires the dataset annotated with bounding boxes. Compared with the point annotations in mainstream object counting issues, the coordinate box-level annotations are …
Department of Computer Science, University of Saskatchewan（萨斯喀彻温大学计算机科学系）；University of Saskatchewan（萨斯喀彻温大学）
In this paper, we propose a simple and effective way to improve one-look regression models for object counting from images. We use class activation map visualizations to illustrate the drawbacks of learning a pure one-look regression model for a counting task. Based on these insights, we enhance one-look regression counting models by regulating activation maps from the final convolution layer of the network with coarse ground-truth activation maps generated from simple dot annotations. We call this strategy heatmap regulation (HR). We show that this simple enhancement effectively suppresses fa…
Current class-agnostic counting methods can generalise to unseen classes but usually require reference images to define the type of object to be counted, as well as instance annotations during training. Reference-less class-agnostic counting is an emerging field that identifies counting as, at its core, a repetition-recognition task. Such methods facilitate counting on a changing set composition. We show that a general feature space with global context can enumerate instances in an image without a prior on the object type present. Specifically, we demonstrate that regression from vision transf…
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Object counting, whose aim is to estimate the number of objects from a given image, is an important and challenging computation task. Significant efforts have been devoted to addressing this problem and achieved great progress, yet counting the number of ground objects from remote sensing images is barely studied. In this paper, we are interested in counting dense objects from remote sensing images. Compared with object counting in a natural scene, this task is challenging in the following factors: large scale variation, complex cluttered background, and orientation arbitrariness. More importa…
Supervised learning is often used to count objects in images, but for counting small, densely located objects, the required image annotations are burdensome to collect. Counting plant organs for image-based plant phenotyping falls within this category. Object counting in plant images is further challenged by having plant image datasets with significant domain shift due to different experimental conditions, e.g. applying an annotated dataset of indoor plant images for use on outdoor images, or on a different plant species. In this paper, we propose a domain-adversarial learning approach for dom…
The crowd counting task aims at estimating the number of people located in an image or a frame from videos. Existing methods widely adopt density maps as the training targets to optimize the point-to-point loss. While in testing phase, we only focus on the differences between the crowd numbers and the global summation of density maps, which indicate the inconsistency between the training targets and the evaluation criteria. To solve this problem, we introduce a new target, named local counting map (LCM), to obtain more accurate results than density map based approaches. Moreover, we also propo…
In this paper we propose a technique to adapt a convolutional neural network (CNN) based object counter to additional visual domains and object types while still preserving the original counting function. Domain-specific normalisation and scaling operators are trained to allow the model to adjust to the statistical distributions of the various visual domains. The developed adaptation technique is used to produce a singular patch-based counting regressor capable of counting various object types including people, vehicles, cell nuclei and wildlife. As part of this study a challenging new cell co…
California Institute of Technology（加州理工学院）；AWS AI Labs（亚马逊云科技人工智能实验室）；Trout Unlimited（鳟鱼保护协会）；Cornell University（康奈尔大学）
We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target…
