SPARC：面向视觉-语言模型的零样本多标签识别的分数提示与自适应融合
SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models
AI总结：
提出SPARC方法，将视觉-语言模型视为黑盒，利用大语言模型生成复合提示并分析分数偏差，通过去偏和分数融合算法提升零样本多标签识别性能，在mAP上优于需训练数据的方法。
摘要：
零样本多标签识别（MLR）在视觉-语言模型（VLM）中，在没有训练数据、模型调优或架构修改的情况下面临重大挑战。现有方法需要提示调优或架构适配，限制了零样本的适用性。我们的工作提出了一种新颖的解决方案，将VLM视为黑盒，利用分数而无需训练数据或真实标签。利用大语言模型对物体共现的洞察，我们引入了基于现实物体组合的复合提示。对这些提示分数的分析揭示了VLM的偏差以及“AND”/“OR”信号的歧义，特别是最大复合分数出人意料地逊于第二高分数。我们通过一种去偏和分数融合算法来解决这些问题，该算法校正图像偏差并澄清VLM的响应行为。我们的方法增强了其他零样本方法，持续改进其结果。实验表明，与需要训练数据的方法相比，我们的方法在平均精度均值（mAP）上表现更优，这是通过精细的物体排序实现的，从而实现了稳健的零样本MLR。
英文摘要：
Zero-shot multi-label recognition (MLR) with Vision-Language Models (VLMs) faces significant challenges without training data, model tuning, or architectural modifications. Existing approaches require prompt tuning or architectural adaptations, limiting zero-shot applicability. Our work proposes a novel solution treating VLMs as black boxes, leveraging scores without training data or ground truth. Using large language model insights on object co-occurrence, we introduce compound prompts grounded in realistic object combinations. Analysis of these prompt scores reveals VLM biases and ``AND''/``OR'' signal ambiguities, notably that maximum compound scores are surprisingly suboptimal compared to second-highest scores. We address these through a debiasing and score-fusion algorithm that corrects image bias and clarifies VLM response behaviors. Our method enhances other zero-shot approaches, consistently improving their results. Experiments show superior mean Average Precision (mAP) compared to methods requiring training data, achieved through refined object ranking for robust zero-shot MLR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
