使用比较描述符增强视觉分类
Enhancing Visual Classification using Comparative Descriptors
AI总结：
针对视觉分类中相似类别易混淆的问题，提出比较描述符方法，通过强调目标类与相似类的独特差异并结合CLIP空间过滤，有效提升了分类准确率与鲁棒性。
摘要：
视觉语言模型（VLMs，如CLIP）在视觉分类任务中的性能，已经通过利用来自大型语言模型（LLMs，包括GPT）的语义知识得到了提升。最近的研究表明，在零样本分类任务中，包含额外线索、高级概念甚至随机字符的描述符通常优于仅使用类别名称的描述符。在许多分类任务中，虽然top-1准确率可能相对较低，但top-5准确率通常显著更高。这一差距意味着大多数错误分类发生在少数相似的类别之间，突显了模型在区分具有细微差异的类别时的困难。为了应对这一挑战，我们引入了比较描述符的新颖概念。这些描述符强调目标类别相对于其最相似类别的独特特征，从而增强区分度。通过生成这些比较描述符并将其整合到分类框架中，我们优化了语义焦点并提高了分类准确率。一个额外的过滤过程确保这些描述符在CLIP空间中更接近图像嵌入，从而进一步提升性能。我们的方法通过解决细微类间差异这一具体挑战，展示了在视觉分类任务中更高的准确率和鲁棒性。
英文摘要：
The performance of vision-language models (VLMs), such as CLIP, in visual classification tasks, has been enhanced by leveraging semantic knowledge from large language models (LLMs), including GPT. Recent studies have shown that in zero-shot classification tasks, descriptors incorporating additional cues, high-level concepts, or even random characters often outperform those using only the category name. In many classification tasks, while the top-1 accuracy may be relatively low, the top-5 accuracy is often significantly higher. This gap implies that most misclassifications occur among a few similar classes, highlighting the model's difficulty in distinguishing between classes with subtle differences. To address this challenge, we introduce a novel concept of comparative descriptors. These descriptors emphasize the unique features of a target class against its most similar classes, enhancing differentiation. By generating and integrating these comparative descriptors into the classification framework, we refine the semantic focus and improve classification accuracy. An additional filtering process ensures that these descriptors are closer to the image embeddings in the CLIP space, further enhancing performance. Our approach demonstrates improved accuracy and robustness in visual classification tasks by addressing the specific challenge of subtle inter-class differences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
