arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.10444cs.CVcs.LG
收藏

传递类间相关性

Transferring Inter-Class Correlation

University of Electronic Science and Technology of China(电子科技大学);National Key Lab on Communication, University of Electronic Science and Technology of China(电子科技大学通信抗干扰技术国家级重点实验室);NVIDIA Semiconductor Technology (Shanghai) Co., Ltd.(英伟达半导体科技(上海)有限公司)

Hui Wen, Yue Wu, Chenming Yang, Jingjing Li, Yue Zhu, Xu Jiang, Hancong Duan

摘要:

教师-学生(Teacher-Student,T-S)框架在分类任务中得到广泛应用,通过该框架,一个神经网络(学生)的性能可以通过从另一个训练好的神经网络(教师)传递知识来提高。由于知识的传递与师生之间的网络能力和网络结构有关,如何界定有效知识一直是一个悬而未决的问题。为了解决这个问题,我们在输出层设计了一种新的传递知识的方法,基于自注意的类间关联(ICC)映射,并提出了我们的T-S框架,类间关联传递(ICCT)。

英文摘要:

The Teacher-Student (T-S) framework is widely utilized in the classification tasks, through which the performance of one neural network (the student) can be improved by transferring knowledge from another trained neural network (the teacher). Since the transferring knowledge is related to the network capacities and structures between the teacher and the student, how to define efficient knowledge remains an open question. To address this issue, we design a novel transferring knowledge, the Self-Attention based Inter-Class Correlation (ICC) map in the output layer, and propose our T-S framework, Inter-Class Correlation Transfer (ICCT).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑