类别无关计数(Class-Agnostic Counting, CAC)旨在仅借助少量参考示例,准确计数给定图像中的物体。以往实现该目标的方法依赖额外训练,而近期研究表明,可以利用既有的基础模型,尤其是 Segment Anything Model(SAM),通过实例级分割进行计数,从而在无需训练的情况下完成该任务。尽管前景良好,但当前无需训练的方法在性能上仍落后于基于训练的同类方法。在本研究中,我们提出一种直接的无需训练方案,可有效弥合这一性能差距,并作为一个强有力的基线。我们工作的主要贡献在于发现了四项能够提升性能的关键技术。具体而言,我们建议采用超像素算法生成更精确的初始点提示,利用具备更丰富语义知识的图像编码器替代 SAM 编码器来表示候选对象,并采用多尺度机制和直推式原型方案来更新参考示例的表示。通过结合这四项技术,我们的方法相较现有无需训练的方法取得了显著提升,并达到了与基于训练方法相当的性能。
英文摘要:
Class-Agnostic Counting (CAC) seeks to accurately count objects in a given image with only a few reference examples. While previous methods achieving this relied on additional training, recent efforts have shown that it's possible to accomplish this without training by utilizing pre-existing foundation models, particularly the Segment Anything Model (SAM), for counting via instance-level segmentation. Although promising, current training-free methods still lag behind their training-based counterparts in terms of performance. In this research, we present a straightforward training-free solution that effectively bridges this performance gap, serving as a strong baseline. The primary contribution of our work lies in the discovery of four key technologies that can enhance performance. Specifically, we suggest employing a superpixel algorithm to generate more precise initial point prompts, utilizing an image encoder with richer semantic knowledge to replace the SAM encoder for representing candidate objects, and adopting a multiscale mechanism and a transductive prototype scheme to update the representation of reference examples. By combining these four technologies, our approach achieves significant improvements over existing training-free methods and delivers performance on par with training-based ones.
Does it Really Count? Assessing Semantic Grounding in Text-Guided Class-Agnostic Counting
Institute of Information Science and Technologies of the National Research Council (ISTI-CNR)(意大利国家研究理事会信息科学与技术研究所);University of Pisa - Department of Information Engineering(比萨大学信息工程系)
OCCAM: Class-Agnostic, Training-Free, Prior-Free and Multi-Class Object Counting
Computer Science Department, University of Crete(塞萨洛尼基大学计算机科学系);Institute of Computer Science (ICS), Foundation for Research & Technology – Hellas (FORTH)(希腊基础研究与技术研究所计算机科学研究所)
Bootstrapping MLLM for Weakly-Supervised Class-Agnostic Object Counting
College of Electronic and Information Engineering, Tongji University(电子信息工程学院,同济大学);College of Computer Science, Wuhan University(计算机科学学院,武汉大学);College of Computer Science, Tongji University(计算机科学学院,同济大学);State Key Laboratory of Autonomous Intelligent Unmanned Systems(自主智能无人系统国家重点实验室)
A Recipe for CAC: Mosaic-based Generalized Loss for Improved Class-Agnostic Counting
National Yang Ming Chiao Tung University(国立阳明交通大学);University of Michigan(密歇根大学);Research Center of Information Technology Innovation, Academia Sinica(中央研究院信息技术创新研究中心)
A Unified Object Counting Network with Object Occupation Prior
Nanjing University of Information Science and Technology(南京信息工程大学);Nanjing Forestry University(南京林业大学);Australian Institute for Machine Learning(澳大利亚机器学习研究所);the University of Adelaide(阿德莱德大学);Nanjing University of Posts and Telecommunications(南京邮电大学)
Text-promptable Object Counting via Quantity Awareness Enhancement
College of Electronic and Information Engineering, Tongji University(同济大学电子与信息学院);College of Computer Science, Wuhan University(武汉大学计算机学院);College of Computer Science, Tongji University(同济大学计算机学院);State Key Laboratory of Autonomous Intelligent Unmanned Systems(自主智能无人系统国家重点实验室)
Can SAM Count Anything? An Empirical Study on SAM Counting
Shenzhen Institute of Advanced Technology, Chinese Academy of Science(中国科学院深圳先进技术研究院);Harbin Institute of Technology(哈尔滨工业大学);Peng Cheng Laboratory(鹏城实验室)
本文实证评估 SAM 在少样本目标计数任务上的能力,发现未经微调的 SAM 相比现有方法表现不佳,尤其难以处理小型和拥挤目标。
Department of Computer Science University of Verona(维罗纳大学计算机科学系);University of Verona(维罗纳大学);Department of Computer Science(计算机科学系);Verona(维罗纳(此处为城市名,非机构名,原文本中可能存在格式问题,根据提取规则,仅保留明确机构名,此处正确机构应为University of Verona,原Department of Computer Science University of Verona整体为一个机构,故合并输出))
Rank-Aware Agglomeration of Foundation Models for Immunohistochemistry Image Cell Counting
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);School of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);Department of Gastrointestinal Surgery and the Gastric Cancer Center, Zhongshan Hospital, Fudan University(复旦大学中山医院胃肠外科及胃癌中心);School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);The Comprehensive Breast Care Center, The Second Affiliated Hospital of Xi'an Jiaotong University(西安交通大学第二附属医院综合乳腺护理中心);Department of Pathology, The Second Affiliated Hospital of Xi’an Jiaotong University(西安交通大学第二附属医院病理科);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of General Surgery, Zhongshan Hospital (Xiamen), Fudan University(复旦大学中山医院(厦门)外科科)
Counting Beyond Instances: A Benchmark for Group-Individual Object Counting
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Liverpool(利物浦大学)