模块化嵌入重组用于增量学习
Modular Embedding Recomposition for Incremental Learning
AI总结：
提出MoDER模块化框架，通过训练和组合文本专家增强VLM零样本能力，在14个数据集上验证了有效性。
摘要：
预训练视觉-语言模型（VLM）的出现显著改变了持续学习（CL）领域，主要归功于它们的零样本分类能力。这种能力使VLM非常适合现实世界应用，能够在无需适应的情况下对新出现的未见类别实现稳健性能。然而，当下游任务与预训练领域显著偏离时，微调仍然必不可少。先前的持续学习方法主要关注在下游任务的增量微调过程中保持VLM的零样本能力。我们更进一步，设计了一种将保持转化为增强VLM零样本能力的方法。我们的方法名为模块化嵌入重组（MoDER），引入了一个模块化框架，训练多个文本专家，每个专家专门针对一个已见类别，并将它们存储在一个基础中心中。在推理时，对于每个未见类别，我们查询该中心并组合检索到的专家，以合成一个改进分类的细化原型。我们在两个流行的零样本增量协议Class-IL和MTIL上展示了我们方法的有效性，共涉及14个数据集。代码库可在https://github.com/aimagelab/mammoth获取。
英文摘要：
The advent of pre-trained Vision-Language Models (VLMs) has significantly transformed Continual Learning (CL), mainly due to their zero-shot classification abilities. Such proficiency makes VLMs well-suited for real-world applications, enabling robust performance on novel unseen classes without requiring adaptation. However, fine-tuning remains essential when downstream tasks deviate significantly from the pre-training domain. Prior CL approaches primarily focus on preserving the zero-shot capabilities of VLMs during incremental fine-tuning on a downstream task. We take a step further by devising an approach that transforms preservation into enhancement of the zero-shot capabilities of VLMs. Our approach, named MoDular Embedding Recomposition (MoDER), introduces a modular framework that trains multiple textual experts, each specialized in a single seen class, and stores them in a foundational hub. At inference time, for each unseen class, we query the hub and compose the retrieved experts to synthesize a refined prototype that improves classification. We show the effectiveness of our method across two popular zero-shot incremental protocols, Class-IL and MTIL, comprising a total of 14 datasets. The codebase is available at https://github.com/aimagelab/mammoth.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
