Attn-Adapter：注意力是视觉-语言模型在线少样本学习所需的全部
Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
AI总结：
针对视觉-语言模型少样本学习中离线微调计算量大、易过拟合的问题，提出基于双重注意力机制的在线少样本框架Attn-Adapter，无需重训基础模型即可动态适配，在跨类别、跨数据集泛化上优于SOTA且推理高效。
摘要：
对比式视觉-语言模型在零样本图像识别中表现优异，但在少样本场景下面临挑战：现有方法依赖计算密集的提示学习离线微调，存在过拟合风险。为克服这些局限，我们提出Attn-Adapter，一种新型在线少样本学习框架，通过双重注意力机制增强CLIP（对比语言-图像预训练模型）的适应性。我们的设计通过两个组件融入数据集特定信息：记忆注意力适配器（Memory Attn-Adapter）利用支持样本优化类别嵌入，局部-全局注意力适配器（Local-Global Attn-Adapter）通过整合局部与全局特征丰富图像嵌入。该架构无需重新训练基础模型，仅需少量标注样本即可实现动态适配。Attn-Adapter在跨类别和跨数据集泛化任务上优于现有最优方法，同时保持高效推理，且可适配不同规模的CLIP骨干网络。
英文摘要：
Contrastive vision-language models excel in zero-shot image recognition but face challenges in few-shot scenarios due to computationally intensive offline fine-tuning using prompt learning, which risks overfitting. To overcome these limitations, we propose Attn-Adapter, a novel online few-shot learning framework that enhances CLIP's adaptability via a dual attention mechanism. Our design incorporates dataset-specific information through two components: the Memory Attn-Adapter, which refines category embeddings using support examples, and the Local-Global Attn-Adapter, which enriches image embeddings by integrating local and global features. This architecture enables dynamic adaptation from a few labeled samples without retraining the base model. Attn-Adapter outperforms state-of-the-art methods in cross-category and cross-dataset generalization, maintaining efficient inference and scaling across CLIP backbones.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
