最小化嵌入失真以实现稳健的分布外性能
Minimizing Embedding Distortion for Robust Out-of-Distribution Performance
AI总结:
针对基础模型微调后泛化能力下降的问题,提出可集成到任意任务微调的similarity loss方法,通过最小化嵌入失真平衡任务适配与泛化能力,在卫星图像分类、人脸识别的OOD场景中显著提升性能且保持ID性能。
摘要:
在海量多样化数据集上训练的基础模型,已展现出在不同领域和分布间完成各类零样本任务的卓越泛化能力。本研究旨在解决通过微调让基础模型适配特定下游任务时,如何保留其强大泛化能力的难题。为此,我们提出一种名为“similarity loss(相似性损失)”的新方法,可集成到任意任务的微调流程中。该方法通过最小化微调后嵌入与预训练嵌入之间的失真,在任务特定适配与保留广泛泛化能力间取得平衡。我们在两类不同任务上评估了该方法:卫星图像的图像分类任务与人脸识别任务,聚焦开放类别和域偏移场景以评估分布外(OOD)性能。实验表明,该方法在保持强劲分布内(ID)性能的同时,显著提升了OOD性能。
英文摘要:
Foundational models, trained on vast and diverse datasets, have demonstrated remarkable capabilities in generalizing across different domains and distributions for various zero-shot tasks. Our work addresses the challenge of retaining these powerful generalization capabilities when adapting foundational models to specific downstream tasks through fine-tuning. To this end, we introduce a novel approach we call "similarity loss", which can be incorporated into the fine-tuning process of any task. By minimizing the distortion of fine-tuned embeddings from the pre-trained embeddings, our method strikes a balance between task-specific adaptation and preserving broad generalization abilities. We evaluate our approach on two diverse tasks: image classification on satellite imagery and face recognition, focusing on open-class and domain shift scenarios to assess out-of-distribution (OOD) performance. We demonstrate that this approach significantly improves OOD performance while maintaining strong in-distribution (ID) performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。