面向计算可行的深度主动学习
Towards Computationally Feasible Deep Active Learning
摘要:
主动学习(AL)是一种用于减少训练机器学习模型所需的注释工作的突出技术。深度学习为在实践中部署人工智能的几个基本障碍提供了解决方案,但也引入了许多其他障碍。这些问题之一是训练获取模型并估计其在未标记池中的实例上的不确定性所需的过多计算资源。我们提出了两种技术来解决这个问题的文本分类和标记任务,提供了一个大幅度减少AL迭代持续时间和计算开销的AL中的深度采集模型。我们还证明了我们的算法,利用伪标签和蒸馏模型克服了一个基本的障碍,以前在文献中发现。也就是说,它表明,由于用于在AL期间选择实例的采集模型和在标记数据上训练的后继模型之间的差异,AL的好处可能会减少。我们表明,我们的算法,尽管使用一个更小,更快的采集模型,是能够训练一个更具表现力的继任者模型具有更高的性能。
英文摘要:
Active learning (AL) is a prominent technique for reducing the annotation effort required for training machine learning models. Deep learning offers a solution for several essential obstacles to deploying AL in practice but introduces many others. One of such problems is the excessive computational resources required to train an acquisition model and estimate its uncertainty on instances in the unlabeled pool. We propose two techniques that tackle this issue for text classification and tagging tasks, offering a substantial reduction of AL iteration duration and the computational overhead introduced by deep acquisition models in AL. We also demonstrate that our algorithm that leverages pseudo-labeling and distilled models overcomes one of the essential obstacles revealed previously in the literature. Namely, it was shown that due to differences between an acquisition model used to select instances during AL and a successor model trained on the labeled data, the benefits of AL can diminish. We show that our algorithm, despite using a smaller and faster acquisition model, is capable of training a more expressive successor model with higher performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。