arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.01334cs.LG
收藏

基于不确定性的神经网络文本分类器的有效调节

Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers

University of Hamburg(汉堡大学)

Jakob Smedegaard Andersen, Walid Maalej

摘要:

为了最大限度地提高准确性和提高文本分类器的整体接受度,我们提出了一个有效的框架,在操作中调节分类器的输出。我们的框架侧重于现代神经网络分类器的F1分数(约90%,在实践中仍不适用。我们提出了一种半自动化的方法,使用预测的不确定性,通过不自信,可能是不正确的分类人类主持人。为了最大限度地减少工作量,我们将人工调节的数据限制在准确性增益饱和并且进一步的人工努力不会导致实质性改进的程度。基于三个不同数据集和三个最先进的分类器的一系列基准测试实验表明,我们的框架可以将分类F1分数提高5.1%至11.2%(高达约10%)。98%至99%),同时与随机适度相比,将适度负载降低至73.3%。

英文摘要:

To maximize the accuracy and increase the overall acceptance of text classifiers, we propose a framework for the efficient, in-operation moderation of classifiers' output. Our framework focuses on use cases in which F1-scores of modern Neural Networks classifiers (ca.~90%) are still inapplicable in practice. We suggest a semi-automated approach that uses prediction uncertainties to pass unconfident, probably incorrect classifications to human moderators. To minimize the workload, we limit the human moderated data to the point where the accuracy gains saturate and further human effort does not lead to substantial improvements. A series of benchmarking experiments based on three different datasets and three state-of-the-art classifiers show that our framework can improve the classification F1-scores by 5.1 to 11.2% (up to approx.~98 to 99%), while reducing the moderation load up to 73.3% compared to a random moderation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑