文本与图像互惠互利：利用CLIP增强免训练少样本分类
Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP
AI总结：
提出TIMO方法，通过图像引导文本和文本引导图像相互优化，解决CLIP免训练少样本分类中的异常匹配和文本提示质量问题，超越现有方法。
摘要：
对比语言-图像预训练（CLIP）已广泛应用于视觉任务。值得注意的是，CLIP在少样本学习（FSL）中展现出良好的性能。然而，现有的基于CLIP的免训练FSL方法（即无需额外训练）主要独立学习不同模态，导致两个关键问题：1）图像模态中严重的异常匹配；2）生成的文本提示质量参差不齐。为解决这些问题，我们构建了一种相互引导机制，引入图像引导文本（IGT）组件，通过图像表示修正质量参差不齐的文本提示，以及文本引导图像（TGI）组件，通过文本表示缓解图像模态的异常匹配。通过整合IGT和TGI，我们采用文本-图像相互引导优化的视角，提出TIMO。大量实验表明，TIMO显著优于最先进的（SOTA）免训练方法。此外，通过探索相互引导的程度，我们提出了一种增强变体TIMO-S，其甚至以约100倍更少的时间成本超越最佳需训练方法0.33%。我们的代码可在https://github.com/lyymuwu/TIMO获取。
英文摘要：
Contrastive Language-Image Pretraining (CLIP) has been widely used in vision tasks. Notably, CLIP has demonstrated promising performance in few-shot learning (FSL). However, existing CLIP-based methods in training-free FSL (i.e., without the requirement of additional training) mainly learn different modalities independently, leading to two essential issues: 1) severe anomalous match in image modality; 2) varying quality of generated text prompts. To address these issues, we build a mutual guidance mechanism, that introduces an Image-Guided-Text (IGT) component to rectify varying quality of text prompts through image representations, and a Text-Guided-Image (TGI) component to mitigate the anomalous match of image modality through text representations. By integrating IGT and TGI, we adopt a perspective of Text-Image Mutual guidance Optimization, proposing TIMO. Extensive experiments show that TIMO significantly outperforms the state-of-the-art (SOTA) training-free method. Additionally, by exploring the extent of mutual guidance, we propose an enhanced variant, TIMO-S, which even surpasses the best training-required methods by 0.33% with approximately 100 times less time cost. Our code is available at https://github.com/lyymuwu/TIMO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
