arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.07294cs.LGcs.AIcs.CV
收藏

加强自我训练方法

Enhancing Self-Training Methods

Ohio State University(俄亥俄州立大学);AFRL/RYAP, Wright-Patterson AFB(赖特-帕特森空军基地空军研究实验室/RYAP)

Aswathnarayan Radhakrishnan, Jim Davis, Zachary Rabin, Benjamin Lewis, Matthew Scherreik, Roman Ilin

摘要:

半监督学习方法对小的标记数据集以及大的未标记数据集进行训练。自我训练是一种半监督的教师-学生方法,通常会遇到“确认偏差”的问题,当学生模型反复过度拟合教师模型为未标记数据给出的不正确伪标签时,就会出现这种问题。这种偏差阻碍了自训练迭代中伪标签准确性的提高,导致在几次迭代后模型性能出现不必要的饱和。在这项工作中,我们描述了多个增强功能,以改善自我训练管道,以减轻确认偏差的影响。我们在多个数据集上评估了我们的增强功能,显示了比现有的自我训练设计选择更好的性能。最后,我们还研究了我们的增强方法的可扩展性开放集未标记的数据(包含类中没有看到标记的数据)。

英文摘要:

Semi-supervised learning approaches train on small sets of labeled data along with large sets of unlabeled data. Self-training is a semi-supervised teacher-student approach that often suffers from the problem of "confirmation bias" that occurs when the student model repeatedly overfits to incorrect pseudo-labels given by the teacher model for the unlabeled data. This bias impedes improvements in pseudo-label accuracy across self-training iterations, leading to unwanted saturation in model performance after just a few iterations. In this work, we describe multiple enhancements to improve the self-training pipeline to mitigate the effect of confirmation bias. We evaluate our enhancements over multiple datasets showing performance gains over existing self-training design choices. Finally, we also study the extendability of our enhanced approach to Open Set unlabeled data (containing classes not seen in labeled data).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑