深度主动分类中的抽样偏差:一项实证研究
Sampling Bias in Deep Active Classification: An Empirical Study
摘要:
数据标记和模型训练所需的爆炸性成本和时间是在大型数据集上训练DNN模型的瓶颈。使用主动学习等策略识别较小的代表性数据样本可以帮助缓解此类瓶颈。NLP中的主动学习以前的工作确定了基于不确定性的查询所获得的样本中的采样偏差问题,并开发了昂贵的方法来解决它。使用大型实证研究,我们证明了使用深度模型(如FastText.zip(FTZ))的后验熵的主动集选择对采样偏差和各种算法选择(查询大小和策略)具有鲁棒性,这与传统文献所建议的不同。我们还表明,基于FTZ的查询策略产生的样本集类似于那些更复杂的方法(例如集成网络)。最后,我们通过创建微小的高质量数据集,并利用它们快速廉价地训练大型模型,来展示所选样本的有效性。基于上述,我们提出了一个简单的基线深度主动文本分类,优于国家的最先进的。我们希望所提出的工作是有用的和信息的数据集压缩和涉及主动,半监督或在线学习的情况下的问题。代码和模型可在https://github.com/drimpossible/Sampling-Bias-Active-Learning上获得
英文摘要:
The exploding cost and time needed for data labeling and model training are bottlenecks for training DNN models on large datasets. Identifying smaller representative data samples with strategies like active learning can help mitigate such bottlenecks. Previous works on active learning in NLP identify the problem of sampling bias in the samples acquired by uncertainty-based querying and develop costly approaches to address it. Using a large empirical study, we demonstrate that active set selection using the posterior entropy of deep models like FastText.zip (FTZ) is robust to sampling biases and to various algorithmic choices (query size and strategies) unlike that suggested by traditional literature. We also show that FTZ based query strategy produces sample sets similar to those from more sophisticated approaches (e.g ensemble networks). Finally, we show the effectiveness of the selected samples by creating tiny high-quality datasets, and utilizing them for fast and cheap training of large models. Based on the above, we propose a simple baseline for deep active text classification that outperforms the state-of-the-art. We expect the presented work to be useful and informative for dataset compression and for problems involving active, semi-supervised or online learning scenarios. Code and models are available at: https://github.com/drimpossible/Sampling-Bias-Active-Learning
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。