探索微调中的表征不变性
Exploring Representation Invariance in Finetuning
AI总结：
针对基础模型微调时表征逐渐消失、泛化能力下降的问题，提出表征不变性微调（RIFT）正则化方法，利用流形正交不变性高效最大化预训练与微调模型的表征相似度，兼容主流微调方法，在保持性能的同时更好保留泛化能力。
摘要：
在大规模自然图像上预训练的基础模型被广泛适配于各类跨域低资源下游任务，这得益于其表征所捕捉的可泛化、可迁移模式。然而，后续研究发现这些表征会在微调过程中逐渐消失，伴随而来的是模型原有泛化能力的下降。本文提出，这类任务可以在不损失预训练表征优势的前提下实现有效适配。我们通过引入**表征不变性微调（Representation Invariance FineTuning, RIFT）**来实现这一目标，这是一种正则化方法，它利用流形的正交不变性，以计算高效的方式最大化预训练模型与微调模型之间的表征相似度。实验表明，我们的方法与主流微调方法兼容，能取得具有竞争力甚至更优的性能，同时能更好地保留模型的泛化能力。
英文摘要：
Foundation models pretrained on large-scale natural images are widely adapted to various cross-domain low-resource downstream tasks, benefiting from generalizable and transferable patterns captured by their representations. However, these representations are later found to gradually vanish during finetuning, accompanied by a degradation of model's original generalizability. In this paper, we argue that such tasks can be effectively adapted without sacrificing the benefits of pretrained representations. We approach this by introducing \textit{Representation Invariance FineTuning (RIFT)}, a regularization that maximizes the representation similarity between pretrained and finetuned models by leveraging orthogonal invariance of manifolds in a computationally efficient way. Experiments demonstrate that our method is compatible with mainstream finetuning methods, offering competitive or even enhanced performance and better preservation of the generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
