arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.04751cs.CVcs.AI
收藏

多模态参数高效少样本类增量学习

Multimodal Parameter-Efficient Few-Shot Class Incremental Learning

Neuraptic AI;Public University of Navarra(纳瓦拉公立大学)

Marco D'Alessandro, Alberto Alonso, Enrique Calabrés, Mikel Galar

AI总结:

针对少样本类增量学习中新类过拟合与性能差距问题,提出多模态参数高效方法CPE-CLIP,利用CLIP预训练知识及语言和视觉编码器的可学习提示实现跨会话迁移学习,并引入提示正则化防止遗忘,显著提升性能且大幅降低参数与训练成本。

摘要:

少样本类增量学习(FSCIL)是一项具有挑战性的持续学习任务,在多次学习会话中只有有限的训练样本可用。要在此任务中取得成功,必须避免由少样本训练集中分布偏差所导致的新类过拟合。解决此问题的常规方法是通过添加特殊模块以向后兼容旧类,从而增强预定义骨干架构的表示能力。然而,该方法尚未解决在确保长期高分类准确率的同时,缩小在较大训练集与较小训练集上所获性能之间差距的困境。在本研究中,我们提出了一种名为 Continual Parameter-Efficient CLIP (CPE-CLIP) 的替代方法,以减少不同学习会话间的信息损失。我们不再通过适配额外模块来解决信息损失,而是利用 CLIP 在大规模预训练中获取的丰富知识及其对新概念的有效泛化能力。我们的方法具有多模态和参数高效的特点,依赖于语言和视觉编码器的可学习提示,以实现跨会话的迁移学习。我们还引入了提示正则化来提升性能并防止遗忘。实验结果表明,与最先进的方法相比,CPE-CLIP 显著提升了 FSCIL 性能,同时大幅减少了可学习参数数量和训练成本。

英文摘要:

Few-Shot Class Incremental Learning (FSCIL) is a challenging continual learning task, where limited training examples are available during several learning sessions. To succeed in this task, it is necessary to avoid over-fitting new classes caused by biased distributions in the few-shot training sets. The general approach to address this issue involves enhancing the representational capability of a pre-defined backbone architecture by adding special modules for backward compatibility with older classes. However, this approach has not yet solved the dilemma of ensuring high classification accuracy over time while reducing the gap between the performance obtained on larger training sets and the smaller ones. In this work, we propose an alternative approach called Continual Parameter-Efficient CLIP (CPE-CLIP) to reduce the loss of information between different learning sessions. Instead of adapting additional modules to address information loss, we leverage the vast knowledge acquired by CLIP in large-scale pre-training and its effectiveness in generalizing to new concepts. Our approach is multimodal and parameter-efficient, relying on learnable prompts for both the language and vision encoders to enable transfer learning across sessions. We also introduce prompt regularization to improve performance and prevent forgetting. Our experimental results demonstrate that CPE-CLIP significantly improves FSCIL performance compared to state-of-the-art proposals while also drastically reducing the number of learnable parameters and training costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑