arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.12200cs.CLcs.AI
收藏

一种基于神经跨度的持续命名实体识别模型

A Neural Span-Based Continual Named Entity Recognition Model

Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室)

Yunan Zhang, Qingcai Chen

AI总结:

针对持续命名实体识别中旧知识遗忘与新类型冲突问题,提出SpanKL,融合跨度建模、知识蒸馏和多标签预测,在OntoNotes与Few-NERD增量基准上显著优于现有SoTA。

摘要:

具备持续学习(CL)能力的命名实体识别(NER)模型在实体类型不断增加的领域(如个人助理)具有现实价值。同时,NER的学习范式也发展到基于跨度的方法等新模式。然而,其在CL方面的潜力尚未得到充分探索。本文提出SpanKL,这是一种简单而有效的基于跨度的模型,采用知识蒸馏(KD)保留记忆,并通过多标签预测防止CL-NER中的冲突。与以往的序列标注方法不同,SpanKL在跨度和实体层面的内在独立建模以及所设计的连贯优化,促进了其在每个增量步骤中的学习并缓解了遗忘。在源自OntoNotes和Few-NERD的合成CL数据集上的实验表明,SpanKL在许多方面显著优于以往的SoTA,并获得了CL与上界之间的最小差距,揭示了其较高的实践价值。代码可在https://github.com/Qznan/SpanKL获取。

英文摘要:

Named Entity Recognition (NER) models capable of Continual Learning (CL) are realistically valuable in areas where entity types continuously increase (e.g., personal assistants). Meanwhile the learning paradigm of NER advances to new patterns such as the span-based methods. However, its potential to CL has not been fully explored. In this paper, we propose SpanKL, a simple yet effective Span-based model with Knowledge distillation (KD) to preserve memories and multi-Label prediction to prevent conflicts in CL-NER. Unlike prior sequence labeling approaches, the inherently independent modeling in span and entity level with the designed coherent optimization on SpanKL promotes its learning at each incremental step and mitigates the forgetting. Experiments on synthetic CL datasets derived from OntoNotes and Few-NERD show that SpanKL significantly outperforms previous SoTA in many aspects, and obtains the smallest gap from CL to the upper bound revealing its high practiced value. The code is available at https://github.com/Qznan/SpanKL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑