arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.01967cs.CV
收藏

利用局部表示力提升少样本分类

Harnessing the Power of Local Representations for Few-Shot Classification

Shi Tang, Guiming Luo, Xinchen Ye, Zhiyi Xia

AI总结:

本文提出通过改进的预训练方法和适应性度量机制,提升少样本分类中对新类别的泛化能力,取得新state-of-the-art成果。

摘要:

泛化到训练时未见过的类别是少样本分类的关键挑战。最近的度量方法试图通过局部表示来解决这个问题,但它们无法充分利用这些表示,原因包括(i)预训练特征提取器的不当监督,以及(ii)度量在处理各种可能的局部特征集组合时缺乏适应性。在本工作中,我们利用局部表示来改进新类别的泛化能力。对于特征提取器,我们设计了一种新的预训练范式,通过软标签学习随机裁剪的补丁。它利用补丁的类级多样性,同时减少其语义错位对硬标签的影响。为了使网络输出与软标签对齐,我们还提出了一种UniCon KL-Divergence,强调每个基础类在描述“非基础”补丁中的等贡献。对于度量,我们将测量局部特征集表述为熵正则化的最优传输问题,以引入处理由同质元素组成的集合的能力。此外,我们设计了一个Modulate模块,使度量具备必要的适应性。我们的方法在三个流行的基准上取得了新的state-of-the-art性能。此外,在细粒度场景中,它超过了state-of-the-art的转导和跨模态方法。

英文摘要:

Generalizing to novel classes unseen during training is a key challenge of few-shot classification. Recent metric-based methods try to address this by local representations. However, they are unable to take full advantage of them due to (i) improper supervision for pretraining the feature extractor, and (ii) lack of adaptability in the metric for handling various possible compositions of local feature sets. In this work, we harness the power of local representations in improving novel-class generalization. For the feature extractor, we design a novel pretraining paradigm that learns randomly cropped patches by soft labels. It utilizes the class-level diversity of patches while diminishing the impact of their semantic misalignments to hard labels. To align network output with soft labels, we also propose a UniCon KL-Divergence that emphasizes the equal contribution of each base class in describing "non-base" patches. For the metric, we formulate measuring local feature sets as an entropy-regularized optimal transport problem to introduce the ability to handle sets consisting of homogeneous elements. Furthermore, we design a Modulate Module to endow the metric with the necessary adaptability. Our method achieves new state-of-the-art performance on three popular benchmarks. Moreover, it exceeds state-of-the-art transductive and cross-modal methods in the fine-grained scenario.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑