用于无监督图像分类的对比知识聚合
Contrastive Knowledge Amalgamation for Unsupervised Image Classification
AI总结：
提出对比知识聚合（CKA）框架，通过引入对比损失和对齐损失实现类内凝聚与类间分离，使学生模型能有效从异构教师模型学习无监督图像分类任务。
摘要：
知识聚合（KA）旨在学习一个紧凑的学生模型，以处理来自各自专门负责不同任务的多个教师模型的联合目标。当前方法侧重于在公共表示空间中粗略地对齐教师和学生，使学生难以从一组异构教师中学习到合适的决策边界。此外，以往工作中的KL散度仅最小化教师和学生之间的概率分布差异，忽略了教师的内在特征。因此，我们提出了一种新颖的对比知识聚合（CKA）框架，引入对比损失和对齐损失以实现类内凝聚和类间分离。模型内和模型间的对比损失旨在扩大不同类别表示之间的距离。引入对齐损失以最小化公共表示空间中师生模型的样本级分布差异。此外，在任务级聚合中，学生通过软目标高效且灵活地学习异构无监督分类任务。在基准上的大量实验证明了CKA在特定任务及多任务聚合中的泛化能力。全面的消融研究进一步深入揭示了我们的CKA。
英文摘要：
Knowledge amalgamation (KA) aims to learn a compact student model to handle the joint objective from multiple teacher models that are are specialized for their own tasks respectively. Current methods focus on coarsely aligning teachers and students in the common representation space, making it difficult for the student to learn the proper decision boundaries from a set of heterogeneous teachers. Besides, the KL divergence in previous works only minimizes the probability distribution difference between teachers and the student, ignoring the intrinsic characteristics of teachers. Therefore, we propose a novel Contrastive Knowledge Amalgamation (CKA) framework, which introduces contrastive losses and an alignment loss to achieve intra-class cohesion and inter-class separation.Contrastive losses intra- and inter- models are designed to widen the distance between representations of different classes. The alignment loss is introduced to minimize the sample-level distribution differences of teacher-student models in the common representation space.Furthermore, the student learns heterogeneous unsupervised classification tasks through soft targets efficiently and flexibly in the task-level amalgamation. Extensive experiments on benchmarks demonstrate the generalization capability of CKA in the amalgamation of specific task as well as multiple tasks. Comprehensive ablation studies provide a further insight into our CKA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
