LightNER：一种通过可插拔编程实现低资源NER的轻量级调优范例
LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting
摘要：
大多数NER方法依赖于大量的标记数据进行模型训练，这在具有有限训练数据的低资源场景中很难实现。现有的主流方法通常会遇到目标域与资源丰富的源域具有不同的标签集的挑战，这可以归纳为类迁移和域迁移。在本文中，我们提出了一个轻量级的调优范式低资源NER通过可插拔提示（LightNER）。具体来说，我们构建了统一的可学习的实体类别的动词生成的实体跨度序列和实体类别没有任何标签特定的分类器，从而解决了类迁移问题。我们进一步提出了一个可插入的指导模块，通过将可学习的参数作为指导纳入自我注意力层，它可以重新调节注意力并适应预先训练的权重。请注意，我们只调整那些插入的模块，而预训练的语言模型的整个参数是固定的，因此，使我们的方法对于低资源场景是轻量级和灵活的，并且可以更好地跨领域传输知识。实验结果表明，LightNER在标准监督环境中可以获得相当的性能，并在低资源环境中优于强基线。代码在https://github.com/zjunlp/DeepKE/tree/main/example/ner/few-shot中。
英文摘要：
Most NER methods rely on extensive labeled data for model training, which struggles in the low-resource scenarios with limited training data. Existing dominant approaches usually suffer from the challenge that the target domain has different label sets compared with a resource-rich source domain, which can be concluded as class transfer and domain transfer. In this paper, we propose a lightweight tuning paradigm for low-resource NER via pluggable prompting (LightNER). Specifically, we construct the unified learnable verbalizer of entity categories to generate the entity span sequence and entity categories without any label-specific classifiers, thus addressing the class transfer issue. We further propose a pluggable guidance module by incorporating learnable parameters into the self-attention layer as guidance, which can re-modulate the attention and adapt pre-trained weights. Note that we only tune those inserted module with the whole parameter of the pre-trained language model fixed, thus, making our approach lightweight and flexible for low-resource scenarios and can better transfer knowledge across domains. Experimental results show that LightNER can obtain comparable performance in the standard supervised setting and outperform strong baselines in low-resource settings. Code is in https://github.com/zjunlp/DeepKE/tree/main/example/ner/few-shot.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
