英、法、西班牙语下的小样本临床实体识别：掩码语言模型优于生成式模型提示学习
Few-shot clinical entity recognition in English, French and Spanish: masked language models outperform generative model prompting
AI总结：
本研究针对临床领域小样本命名实体识别任务，对比生成式大模型提示学习与掩码模型微调方法，发现轻量掩码biLSTM-CRF模型在英、法、西三语临床NER上F1更优且环境影响更低，提示当前LLM提示学习暂不适合临床NER生产。
摘要：
大语言模型（LLM）已成为众多自然语言处理任务的首选方案。在专业领域这类低资源环境中，人们期望其小样本能力能带来高性能表现。命名实体识别（NER）是信息抽取中的关键任务，却未被纳入近期的LLM基准测试。我们需要更好地理解LLM在包括非英语在内的多种场景下的NER性能。本研究旨在评估通过提示工程使用的生成式LLM在小样本临床NER任务中的表现。我们在覆盖英语、法语和西班牙语的14个NER数据集上，对比了13个采用提示学习的自回归模型与16个采用微调的掩码模型。基于提示的自回归模型在通用NER任务上取得了有竞争力的F1值，但在临床领域中，性能不及基于掩码模型的更轻量的biLSTM-CRF标注器。此外，与自回归模型相比，掩码模型的环境影响更低。研究发现在所研究的三种语言中具有一致性，这表明LLM提示学习目前还不适用于临床领域的NER生产场景。
英文摘要：
Large language models (LLMs) have become the preferred solution for many natural language processing tasks. In low-resource environments such as specialized domains, their few-shot capabilities are expected to deliver high performance. Named Entity Recognition (NER) is a critical task in information extraction that is not covered in recent LLM benchmarks. There is a need for better understanding the performance of LLMs for NER in a variety of settings including languages other than English. This study aims to evaluate generative LLMs, employed through prompt engineering, for few-shot clinical NER. %from the perspective of F1 performance and environmental impact. We compare 13 auto-regressive models using prompting and 16 masked models using fine-tuning on 14 NER datasets covering English, French and Spanish. While prompt-based auto-regressive models achieve competitive F1 for general NER, they are outperformed within the clinical domain by lighter biLSTM-CRF taggers based on masked models. Additionally, masked models exhibit lower environmental impact compared to auto-regressive models. Findings are consistent across the three languages studied, which suggests that LLM prompting is not yet suited for NER production in the clinical domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
