论分类中的迁移:类别子集的泛化能力如何?
On Transfer in Classification: How Well do Subsets of Classes Generalize?
AI总结:
本研究针对分类任务中类别子集间迁移的理论基础缺失问题,构建了类别子集偏序集的可迁移性理论框架,可预测最优泛化类别子集,还探究了小样本学习场景,助力理解迁移机制与模型泛化。
摘要:
在分类任务中,通常可观察到在给定类别集上训练的模型能够泛化到此前未见过的类别,这表明模型具备超越初始任务的学习能力。这种能力常被应用于迁移学习场景中——即预训练模型可在经过微调或不经微调的情况下,用于处理新类别。令人意外的是,目前仅有极少论文探究这一现象背后的理论根源。本研究致力于为类别集合间的可迁移性构建此类理论框架的基础。具体而言,我们建立了一个类别子集的偏序集,该工具可用于表征哪些类别子集能够泛化到其他类别子集。在更贴近实际的场景中,我们探究了该框架的预测能力:即当在全部类别上测试时,哪些类别子集能够带来最优性能。我们还对小样本学习(few-shot learning)场景展开了探究,而迁移正是该场景的核心评判标准。本研究有助于加深对迁移机制与模型泛化能力的理解。
英文摘要:
In classification, it is usual to observe that models trained on a given set of classes can generalize to previously unseen ones, suggesting the ability to learn beyond the initial task. This ability is often leveraged in the context of transfer learning where a pretrained model can be used to process new classes, with or without fine tuning. Surprisingly, there are a few papers looking at the theoretical roots beyond this phenomenon. In this work, we are interested in laying the foundations of such a theoretical framework for transferability between sets of classes. Namely, we establish a partially ordered set of subsets of classes. This tool allows to represent which subset of classes can generalize to others. In a more practical setting, we explore the ability of our framework to predict which subset of classes can lead to the best performance when testing on all of them. We also explore few-shot learning, where transfer is the golden standard. Our work contributes to better understanding of transfer mechanics and model generalization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。