arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.14541cs.CL
收藏

无灾难性遗忘的持续命名实体识别

Continual Named Entity Recognition without Catastrophic Forgetting

Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Baidu Inc(百度公司);Hong Kong Baptist University(香港浸会大学);Australian Artificial Intelligence Institute, University of Technology Sydney(悉尼科技大学澳大利亚人工智能研究所)

Duzhen Zhang, Wei Cong, Jiahua Dong, Yahan Yu, Xiuyi Chen, Yonggang Zhang, Zhen Fang

AI总结:

本文针对持续命名实体识别中的灾难性遗忘和非实体类型语义偏移问题,提出池化特征蒸馏损失、基于置信度的伪标签及自适应重加权策略,在多个数据集上显著超越现有方法。

摘要:

持续命名实体识别(CNER)是一个新兴领域,涉及通过顺序引入新实体类型来更新现有模型。然而,持续学习方法常受到灾难性遗忘的严重影响。在CNER中,由于每一步将之前步骤的旧实体类型合并为非实体类型,这一问题进一步加剧,导致了所谓的非实体类型的语义偏移问题。本文提出了一种池化特征蒸馏损失,巧妙地在保留旧实体类型知识与学习新类型之间取得平衡,从而更有效地缓解灾难性遗忘问题。此外,我们为非实体类型开发了一种基于置信度的伪标签方法,即使用旧模型预测实体类型以处理非实体类型的语义偏移。在伪标签过程后,我们提出了一种自适应重加权的类型均衡学习策略,以应对类型分布偏斜问题。我们在三个不同数据集的十种CNER设置上进行了全面实验。结果表明,我们的方法显著优于先前的最先进方法,在Micro和Macro F1分数上分别平均提升了6.3%和8.0%。

英文摘要:

Continual Named Entity Recognition (CNER) is a burgeoning area, which involves updating an existing model by incorporating new entity types sequentially. Nevertheless, continual learning approaches are often severely afflicted by catastrophic forgetting. This issue is intensified in CNER due to the consolidation of old entity types from previous steps into the non-entity type at each step, leading to what is known as the semantic shift problem of the non-entity type. In this paper, we introduce a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones, thereby more effectively mitigating the problem of catastrophic forgetting. Additionally, we develop a confidence-based pseudo-labeling for the non-entity type, \emph{i.e.,} predicting entity types using the old model to handle the semantic shift of the non-entity type. Following the pseudo-labeling process, we suggest an adaptive re-weighting type-balanced learning strategy to handle the issue of biased type distribution. We carried out comprehensive experiments on ten CNER settings using three different datasets. The results illustrate that our method significantly outperforms prior state-of-the-art approaches, registering an average improvement of $6.3$\% and $8.0$\% in Micro and Macro F1 scores, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑