一个模型适用于所有领域:面向跨领域命名实体识别的协作式领域前缀调优
One Model for All Domains: Collaborative Domain-Prefix Tuning for Cross-Domain NER
AI总结:
本文提出基于生成式预训练语言模型的协作式领域前缀调优方法 CP-NER,通过冻结模型参数并利用领域指令实现多源知识迁移,在 Cross-NER 基准上提升了单源和多源跨领域 NER 性能。
摘要:
跨领域命名实体识别(NER)是解决实际场景中低资源问题的一项具有挑战性的任务。以往的典型解决方案主要利用预训练语言模型(PLMs),借助资源丰富领域的数据获得一个 NER 模型,并将其适配到目标领域。由于不同领域中实体类型之间存在不匹配问题,以往的方法通常需要调优 PLMs 的全部参数,最终为每个领域都生成一个全新的 NER 模型。此外,当前模型仅关注利用单一通用源领域的知识,而未能成功地将多个源领域的知识迁移到目标领域。为了解决这些问题,我们基于文本到文本的生成式 PLMs,提出了面向跨领域 NER 的协作式领域前缀调优方法(CP-NER)。具体来说,我们提出了文本到文本生成方式,以领域相关的指令为基础,在不进行结构修改的情况下将知识迁移到新的领域 NER 任务中。我们利用冻结的 PLMs,并进行协作式领域前缀调优,以激发 PLMs 处理不同领域 NER 任务的潜力。在 Cross-NER 基准上的实验结果表明,所提出的方法具有灵活的迁移能力,并且在单源和多源跨领域 NER 任务上均表现更好。代码可在 https://github.com/zjunlp/DeepKE/tree/main/example/ner/cross 获取。
英文摘要:
Cross-domain NER is a challenging task to address the low-resource problem in practical scenarios. Previous typical solutions mainly obtain a NER model by pre-trained language models (PLMs) with data from a rich-resource domain and adapt it to the target domain. Owing to the mismatch issue among entity types in different domains, previous approaches normally tune all parameters of PLMs, ending up with an entirely new NER model for each domain. Moreover, current models only focus on leveraging knowledge in one general source domain while failing to successfully transfer knowledge from multiple sources to the target. To address these issues, we introduce Collaborative Domain-Prefix Tuning for cross-domain NER (CP-NER) based on text-to-text generative PLMs. Specifically, we present text-to-text generation grounding domain-related instructors to transfer knowledge to new domain NER tasks without structural modifications. We utilize frozen PLMs and conduct collaborative domain-prefix tuning to stimulate the potential of PLMs to handle NER tasks across various domains. Experimental results on the Cross-NER benchmark show that the proposed approach has flexible transfer ability and performs better on both one-source and multiple-source cross-domain NER tasks. Codes are available in https://github.com/zjunlp/DeepKE/tree/main/example/ner/cross.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。