arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.14153cs.CVcs.LG
收藏

时间输出离散的半监督主动学习

Semi-Supervised Active Learning with Temporal Output Discrepancy

Baidu Research(百度研究院);Austin Peay State University(奥斯汀佩伊州立大学);Styling AI

Siyu Huang, Tianyang Wang, Haoyi Xiong, Jun Huan, Dejing Dou

摘要:

虽然深度学习在广泛的任务中取得了成功,但它高度依赖于大量注释数据的收集,这是昂贵且耗时的。为了降低数据标注的成本,已经提出了主动学习来交互式地查询oracle以标注未标记数据集中的一小部分信息样本。受损失较高的样本通常比损失较低的样本对模型的信息量更大这一事实的启发,在本文中,我们提出了一种新的深度主动学习方法,当未标记的样本被认为包含高损失时,该方法会查询Oracle的数据注释。我们的方法的核心是一个测量时间输出离散(TOD),估计样本损失通过评估模型在不同的优化步骤给出的输出的差异。我们的理论研究表明,TOD下限的累积样本损失,因此它可以用来选择信息的未标记的样本。在此基础上,进一步提出了一种有效的无标记数据采样策略和一种无监督学习准则,通过引入无标记数据来提高模型性能。由于TOD的简单性,我们的主动学习方法是有效的,灵活的,任务不可知的。大量的实验结果表明,我们的方法取得了优于国家的最先进的主动学习方法的图像分类和语义分割任务的性能。

英文摘要:

While deep learning succeeds in a wide range of tasks, it highly depends on the massive collection of annotated data which is expensive and time-consuming. To lower the cost of data annotation, active learning has been proposed to interactively query an oracle to annotate a small proportion of informative samples in an unlabeled dataset. Inspired by the fact that the samples with higher loss are usually more informative to the model than the samples with lower loss, in this paper we present a novel deep active learning approach that queries the oracle for data annotation when the unlabeled sample is believed to incorporate high loss. The core of our approach is a measurement Temporal Output Discrepancy (TOD) that estimates the sample loss by evaluating the discrepancy of outputs given by models at different optimization steps. Our theoretical investigation shows that TOD lower-bounds the accumulated sample loss thus it can be used to select informative unlabeled samples. On basis of TOD, we further develop an effective unlabeled data sampling strategy as well as an unsupervised learning criterion that enhances model performance by incorporating the unlabeled data. Due to the simplicity of TOD, our active learning approach is efficient, flexible, and task-agnostic. Extensive experimental results demonstrate that our approach achieves superior performances than the state-of-the-art active learning methods on image classification and semantic segmentation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑