SDVPT：面向开放世界目标计数的语义驱动视觉提示微调
SDVPT: Semantic-Driven Visual Prompt Tuning for Open-World Object Counting
AI总结：
提出SDVPT框架，通过两阶段视觉提示学习（CSPI和TGPR）动态合成未见类别的提示，以最小开销提升开放世界目标计数模型的泛化能力，并在FSC-147、CARPK和PUCPR+数据集上验证有效性。
摘要：
开放世界目标计数利用预训练视觉语言模型（VLM）强大的文本-图像对齐能力，能够根据文本查询对图像中的任意类别进行计数。然而，广泛采用的朴素微调策略仅专注于训练中包含类别的文本-图像一致性，导致对未见类别的泛化能力有限。在这项工作中，我们提出了一种即插即用的语义驱动视觉提示微调框架（SDVPT），以极小的参数和推理时间开销将知识从训练集迁移到未见类别。首先，我们引入了一种两阶段视觉提示学习策略，包括类别特定提示初始化（CSPI）和拓扑引导的提示精炼（TGPR）。CSPI生成类别特定的视觉提示，随后TGPR从VLM的文本编码器中提炼潜在的结构模式以精炼这些提示。在推理过程中，我们根据未见类别与训练类别之间的语义相关性动态合成未见类别的视觉提示，从而促进未见类别的稳健文本-图像对齐。将SDVPT与所有可用的开放世界目标计数模型集成的大量实验证明了其在三个广泛使用的数据集（FSC-147、CARPK和PUCPR+）上的有效性和适应性。
英文摘要：
Open-world object counting leverages the robust text-image alignment of pre-trained vision-language models (VLMs) to enable counting of arbitrary categories in images specified by textual queries. However, widely adopted naive fine-tuning strategies concentrate exclusively on text-image consistency for categories contained in training, which leads to limited generalizability for unseen categories. In this work, we propose a plug-and-play Semantic-Driven Visual Prompt Tuning framework (SDVPT) that transfers knowledge from the training set to unseen categories with minimal overhead in parameters and inference time. First, we introduce a two-stage visual prompt learning strategy composed of Category-Specific Prompt Initialization (CSPI) and Topology-Guided Prompt Refinement (TGPR). The CSPI generates category-specific visual prompts, and then TGPR distills latent structural patterns from the VLM's text encoder to refine these prompts. During inference, we dynamically synthesize the visual prompts for unseen categories based on the semantic correlation between unseen and training categories, facilitating robust text-image alignment for unseen categories. Extensive experiments integrating SDVPT with all available open-world object counting models demonstrate its effectiveness and adaptability across three widely used datasets: FSC-147, CARPK, and PUCPR+.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
