arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.08698cs.CLcs.AI
收藏

验证NER数据注释中的标签一致性

Validating Label Consistency in NER Data Annotation

University of Notre Dame(圣母大学);Harbin Institute of Technology(哈尔滨工业大学)

Qingkai Zeng, Mengxia Yu, Wenhao Yu, Tianwen Jiang, Meng Jiang

摘要:

数据注释在确保您的命名实体识别(NER)项目使用正确的信息进行训练方面起着至关重要的作用。由于注释的复杂性,生成最准确的标签是一项挑战。数据注释的多个子集之间的标签不一致(例如,训练集和测试集,或多个训练子集)是标记错误的指示符。在这项工作中,我们提出了一个实证的方法来探讨标签(不)一致性和NER模型性能之间的关系。它可用于验证多个NER数据标注集的标签一致性(或捕获不一致性)。在实验中,我们的方法识别了SCIERC和CoNLL 03数据集中的测试数据的标签不一致性(标签错误率分别为26.7%和5.4%)。它验证了两个数据集的校正版本的一致性。

英文摘要:

Data annotation plays a crucial role in ensuring your named entity recognition (NER) projects are trained with the right information to learn from. Producing the most accurate labels is a challenge due to the complexity involved with annotation. Label inconsistency between multiple subsets of data annotation (e.g., training set and test set, or multiple training subsets) is an indicator of label mistakes. In this work, we present an empirical method to explore the relationship between label (in-)consistency and NER model performance. It can be used to validate the label consistency (or catches the inconsistency) in multiple sets of NER data annotation. In experiments, our method identified the label inconsistency of test data in SCIERC and CoNLL03 datasets (with 26.7% and 5.4% label mistakes). It validated the consistency in the corrected version of both datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑