Meta-Adapter：一种用于视觉-语言模型的在线少样本学习器
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model
AI总结：
本文提出 Meta-Adapter，一种轻量级残差式在线适配器，在少样本引导下细化 CLIP 特征，无需离线微调即可实现高效少样本学习，并在分类、检测和分割任务上取得显著提升。
摘要：
对比式视觉-语言预训练模型 CLIP 在感知开放世界视觉概念方面展现出显著潜力，能够实现有效的零样本图像识别。然而，基于 CLIP 的少样本学习方法通常需要在少样本上对参数进行离线微调，导致推理时间更长，并在某些领域存在过拟合风险。为解决这些挑战，我们提出了 Meta-Adapter，一种轻量级残差式适配器，以在线方式在少样本的引导下细化 CLIP 特征。仅需少量训练样本，我们的方法就能实现有效的少样本学习能力，并无需额外微调即可泛化到未见数据或任务，从而获得有竞争力的性能和高效率。在不使用任何额外技巧的情况下，我们的方法在八个图像分类数据集上比最先进的在线少样本学习方法平均高出 3.6%，且推理速度更快。此外，我们的模型简单灵活，可作为即插即用模块直接应用于下游任务。无需进一步微调，Meta-Adapter 在开放词汇目标检测和分割任务中取得了显著的性能提升。
英文摘要：
The contrastive vision-language pre-training, known as CLIP, demonstrates remarkable potential in perceiving open-world visual concepts, enabling effective zero-shot image recognition. Nevertheless, few-shot learning methods based on CLIP typically require offline fine-tuning of the parameters on few-shot samples, resulting in longer inference time and the risk of over-fitting in certain domains. To tackle these challenges, we propose the Meta-Adapter, a lightweight residual-style adapter, to refine the CLIP features guided by the few-shot samples in an online manner. With a few training samples, our method can enable effective few-shot learning capabilities and generalize to unseen data or tasks without additional fine-tuning, achieving competitive performance and high efficiency. Without bells and whistles, our approach outperforms the state-of-the-art online few-shot learning method by an average of 3.6\% on eight image classification datasets with higher inference speed. Furthermore, our model is simple and flexible, serving as a plug-and-play module directly applicable to downstream tasks. Without further fine-tuning, Meta-Adapter obtains notable performance improvements in open-vocabulary object detection and segmentation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
