arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2004.12126cs.CL
收藏

关于命名实体识别的严谨研究:微调预训练模型能带来希望之地吗?

A Rigorous Study on Named Entity Recognition: Can Fine-tuning Pretrained Model Lead to the Promised Land?

University of Chinese Academy of Sciences(中国科学院大学);Chinese Information Processing Laboratory State Key Laboratory of Computer ScienceInstitute of Software, Chinese Academy of Sciences(中国科学院软件研究所计算机科学国家重点实验室中文信息处理实验室);Huawei Cloud&AI(华为云与人工智能)

Hongyu Lin, Yaojie Lu, Jialong Tang, Xianpei Han, Le Sun, Zhicheng Wei, Nicholas Jing Yuan

摘要:

微调预训练模型在标准NER基准测试中取得了令人满意的性能。一般来说,这些基准具有很强的名称规则性,高提及覆盖率和足够的上下文多样性。不幸的是,当扩大净入学率开放的情况下,这些优势可能不再存在。因此,它提出了一个关键的问题,即以前可信的方法在面对这些挑战时是否仍然有效。由于目前没有可用的数据集来研究这个问题,本文提出了标准基准进行随机化测试。具体来说,我们从基准测试中删除名称规则性,提及覆盖率和上下文多样性,以探索它们对模型泛化能力的影响。为了进一步验证我们的结论,我们还构建了一个新的开放NER数据集,该数据集关注名称规则性较弱和提及覆盖率较低的实体类型,以验证我们的结论。从随机化测试和实证实验中,我们得出以下结论:1)名称规则性对于模型推广到未见过的提及至关重要; 2)高提及覆盖率可能会破坏模型的泛化能力; 3)使用预训练编码器时,上下文模式可能不需要捕获大量数据。

英文摘要:

Fine-tuning pretrained model has achieved promising performance on standard NER benchmarks. Generally, these benchmarks are blessed with strong name regularity, high mention coverage and sufficient context diversity. Unfortunately, when scaling NER to open situations, these advantages may no longer exist. And therefore it raises a critical question of whether previous creditable approaches can still work well when facing these challenges. As there is no currently available dataset to investigate this problem, this paper proposes to conduct randomization test on standard benchmarks. Specifically, we erase name regularity, mention coverage and context diversity respectively from the benchmarks, in order to explore their impact on the generalization ability of models. To further verify our conclusions, we also construct a new open NER dataset that focuses on entity types with weaker name regularity and lower mention coverage to verify our conclusion. From both randomization test and empirical experiments, we draw the conclusions that 1) name regularity is critical for the models to generalize to unseen mentions; 2) high mention coverage may undermine the model generalization ability and 3) context patterns may not require enormous data to capture when using pretrained encoders.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑