arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.11686cs.CV
收藏

基于表示定理的低样本视觉-语言模型自适应综述

A Survey of Low-shot Vision-Language Model Adaptation via Representer Theorem

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)

Kun Ding, Ying Wang, Gaofeng Meng, Shiming Xiang

AI总结:

本综述从表示定理视角提出统一计算框架,整合并推导现有低样本视觉-语言模型自适应方法,揭示其差异与联系,并在RKHS中建模类间相关性提出改进变体,经11个数据集验证有效。

摘要:

预训练视觉-语言基础模型的出现彻底改变了零样本/少样本(即低样本)图像识别领域。在训练数据有限的条件下,需要解决的关键挑战是如何以参数高效的方式微调预训练视觉-语言模型。此前,已有大量方法被提出以应对这一挑战。同时,也发表了一些综述论文来总结这些工作。然而,目前仍然缺乏一个统一的计算框架来整合现有方法、揭示其本质并支持深入比较。因此,本综述首先从表示定理(Representer Theorem)的视角提出了一个统一的计算框架,然后通过特化该框架推导出许多现有方法。随后,进行了比较分析以揭示现有方法之间的差异与联系。基于这些分析,提出了一些改进现有工作的可能变体。作为演示,我们通过在再生核希尔伯特空间(RKHS)中对表示器之间的类间相关性进行建模来扩展现有方法,该方法通过利用核岭回归的闭式解来实现。在11个数据集上进行了大量实验以验证该方法的有效性。在本文最后,我们讨论了局限性并提供了进一步的研究方向。

英文摘要:

The advent of pre-trained vision-language foundation models has revolutionized the field of zero/few-shot (i.e., low-shot) image recognition. The key challenge to address under the condition of limited training data is how to fine-tune pre-trained vision-language models in a parameter-efficient manner. Previously, numerous approaches tackling this challenge have been proposed. Meantime, a few survey papers are also published to summarize these works. However, there still lacks a unified computational framework to integrate existing methods together, identify their nature and support in-depth comparison. As such, this survey paper first proposes a unified computational framework from the perspective of Representer Theorem and then derives many of the existing methods by specializing this framework. Thereafter, a comparative analysis is conducted to uncover the differences and relationships between existing methods. Based on the analyses, some possible variants to improve the existing works are presented. As a demonstration, we extend existing methods by modeling inter-class correlation between representers in reproducing kernel Hilbert space (RKHS), which is implemented by exploiting the closed-form solution of kernel ridge regression. Extensive experiments on 11 datasets are conducted to validate the effectiveness of this method. Toward the end of this paper, we discuss the limitations and provide further research directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑