命名实体识别--是否存在玻璃天花板?
Named Entity Recognition -- Is there a glass ceiling?
摘要:
命名实体识别(NER)的最新发展导致了越来越好的模型。但是,是否存在玻璃天花板?我们知道哪些类型的错误仍然很难甚至不可能纠正吗?在本文中,我们提出了一个国家的最先进的机器学习(ML)方法的错误类型的详细分析。我们的研究揭示了斯坦福大学,CMU,FLAIR,ELMO和BERT模型的优缺点,以及他们共同的局限性。我们还介绍了用于改进注释、训练过程和检查模型质量和稳定性的新技术。所呈现的结果是基于CoNLL 2003年英语数据集。补充材料中附有该数据集和新诊断数据集的错误的新丰富语义注释。
英文摘要:
Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this paper, we present a detailed analysis of the types of errors in state-of-the-art machine learning (ML) methods. Our study reveals the weak and strong points of the Stanford, CMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also introduce new techniques for improving annotation, for training processes and for checking a model's quality and stability. Presented results are based on the CoNLL 2003 data set for the English language. A new enriched semantic annotation of errors for this data set and new diagnostic data sets are attached in the supplementary materials.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。