arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10573cs.CL
收藏

LinkNER:使用不确定性将局部命名实体识别模型链接到大语言模型

LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty

Nankai University(南开大学);Tianjin University of Science and Technology(天津科技大学)

Zhen Zhang, Yuhua Zhao, Hang Gao, Mengting Hu

AI总结:

提出LinkNER框架,通过基于不确定性的链接策略RDC将小型微调NER模型与黑盒大语言模型结合,解决未见实体识别问题,在鲁棒性测试中超越SOTA模型。

摘要:

命名实体识别(NER)是自然语言理解的基础任务,对网页内容分析、搜索引擎和信息检索系统具有直接影响。微调的NER模型在标准NER基准上表现良好。然而,由于微调数据有限且缺乏知识,它在未见实体识别上表现不佳。因此,NER模型在网页相关应用中的可用性和可靠性受到影响。相反,像GPT-4这样的大语言模型(LLMs)拥有广泛的外部知识,但研究表明它们缺乏对NER任务的专长。此外,非公开和大规模的权重使得调优LLMs变得困难。为了解决这些挑战,我们提出了一种将小型微调模型与LLMs结合的框架(LinkNER),以及一种基于不确定性的链接策略RDC,使微调模型能够补充黑盒LLMs,实现更好的性能。我们在标准NER测试集和嘈杂的社交媒体数据集上进行了实验。LinkNER提升了NER任务性能,在鲁棒性测试中显著超越SOTA模型。我们还定量分析了不确定性估计方法、LLMs和上下文学习等关键组件对各种NER任务的影响,提供了具体的网页相关建议。代码可在https://github.com/zhzhengit/LinkNER获取。

英文摘要:

Named Entity Recognition (NER) serves as a fundamental task in natural language understanding, bearing direct implications for web content analysis, search engines, and information retrieval systems. Fine-tuned NER models exhibit satisfactory performance on standard NER benchmarks. However, due to limited fine-tuning data and lack of knowledge, it performs poorly on unseen entity recognition. As a result, the usability and reliability of NER models in web-related applications are compromised. Instead, Large Language Models (LLMs) like GPT-4 possess extensive external knowledge, but research indicates that they lack specialty for NER tasks. Furthermore, non-public and large-scale weights make tuning LLMs difficult. To address these challenges, we propose a framework that combines small fine-tuned models with LLMs (LinkNER) and an uncertainty-based linking strategy called RDC that enables fine-tuned models to complement black-box LLMs, achieving better performance. We experiment with both standard NER test sets and noisy social media datasets. LinkNER enhances NER task performance, notably surpassing SOTA models in robustness tests. We also quantitatively analyze the influence of key components like uncertainty estimation methods, LLMs, and in-context learning on diverse NER tasks, offering specific web-related recommendations. Code is available at https://github.com/zhzhengit/LinkNER.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑