arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.04795cs.LGcs.CV
收藏

多头协同训练的半监督学习

Semi-Supervised Learning with Multi-Head Co-Training

Nanjing University(南京大学)

Mingcai Chen, Yuntao Du, Yi Zhang, Shuwei Qian, Chongjun Wang

AI总结:

针对单视图协同训练需额外分类器且易趋同的问题,提出多头协同训练算法,通过多头结构集成基学习器并利用强弱增强策略引入多样性,以极小额外开销提升半监督学习性能。

摘要:

协同训练(Co-training)从自训练(self-training)扩展而来,是半监督学习的框架之一。在没有特征的自然划分的情况下,单视图协同训练以训练额外分类器为代价,且需要精心设计算法以防止各个分类器相互趋同。为消除这些阻碍单视图协同训练应用的障碍,我们提出了一种简单高效的算法——多头协同训练(Multi-Head Co-Training)。通过将基学习器集成到一个多头结构中,模型仅需极少的额外参数。统一模型中的每个分类头通过“弱增强与强增强”策略与其同伴交互,其中多样性由强数据增强自然引入。因此,所提出的方法通过1)隐式促进多样性,2)仅需少量额外计算开销,促进了单视图协同训练。在标准半监督学习基准上的实证研究展示了多头协同训练的有效性。

英文摘要:

Co-training, extended from self-training, is one of the frameworks for semi-supervised learning. Without natural split of features, single-view co-training works at the cost of training extra classifiers, where the algorithm should be delicately designed to prevent individual classifiers from collapsing into each other. To remove these obstacles which deter the adoption of single-view co-training, we present a simple and efficient algorithm Multi-Head Co-Training. By integrating base learners into a multi-head structure, the model is in a minimal amount of extra parameters. Every classification head in the unified model interacts with its peers through a "Weak and Strong Augmentation" strategy, in which the diversity is naturally brought by the strong data augmentation. Therefore, the proposed method facilitates single-view co-training by 1). promoting diversity implicitly and 2). only requiring a small extra computational overhead. The effectiveness of Multi-Head Co-Training is demonstrated in an empirical study on standard semi-supervised learning benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑