CLIP-Count：迈向文本引导的零样本目标计数
CLIP-Count: Towards Text-Guided Zero-Shot Object Counting
AI总结：
本文提出CLIP-Count，首个文本引导的零样本开放词汇目标计数端到端框架，通过图像块-文本对比损失和分层交互模块对齐视觉-语言特征，在多个计数数据集上取得最先进性能。
摘要：
视觉-语言模型的最新进展展示了卓越的零样本文本-图像匹配能力，该能力可迁移到目标检测和分割等下游任务。然而，将这些模型适配用于目标计数仍然是一项艰巨的挑战。在本研究中，我们首先研究了将视觉-语言模型（VLMs）迁移用于类别无关的目标计数。具体而言，我们提出了CLIP-Count，这是第一个以零样本方式在文本引导下估计开放词汇目标密度图的端到端流水线。为了将文本嵌入与密集视觉特征对齐，我们引入了一种图像块-文本对比损失，引导模型学习用于密集预测的信息丰富的图像块级视觉表示。此外，我们设计了一个分层的图像块-文本交互模块，以在视觉特征的不同分辨率层级之间传播语义信息。得益于对预训练VLMs丰富的图像-文本对齐知识的充分利用，我们的方法能够有效地为感兴趣的目标生成高质量的密度图。在FSC-147、CARPK和ShanghaiTech人群计数数据集上的大量实验证明了所提方法具有最先进的准确性和泛化能力。代码可在以下地址获取：https://github.com/songrise/CLIP-Count。
英文摘要：
Recent advances in visual-language models have shown remarkable zero-shot text-image matching ability that is transferable to downstream tasks such as object detection and segmentation. Adapting these models for object counting, however, remains a formidable challenge. In this study, we first investigate transferring vision-language models (VLMs) for class-agnostic object counting. Specifically, we propose CLIP-Count, the first end-to-end pipeline that estimates density maps for open-vocabulary objects with text guidance in a zero-shot manner. To align the text embedding with dense visual features, we introduce a patch-text contrastive loss that guides the model to learn informative patch-level visual representations for dense prediction. Moreover, we design a hierarchical patch-text interaction module to propagate semantic information across different resolution levels of visual features. Benefiting from the full exploitation of the rich image-text alignment knowledge of pretrained VLMs, our method effectively generates high-quality density maps for objects-of-interest. Extensive experiments on FSC-147, CARPK, and ShanghaiTech crowd counting datasets demonstrate state-of-the-art accuracy and generalizability of the proposed method. Code is available: https://github.com/songrise/CLIP-Count.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
