BioCoref: 基于大语言模型的生物医学共指消解基准测试
BioCoref: Benchmarking Biomedical Coreference Resolution with LLMs
AI总结：
本研究针对生物医学共指消解的独特挑战，以CRAFT语料库为基准，通过多组提示实验评估生成式LLMs的性能并与SpanBERT对比，发现实体增强提示可提升LLaMA等模型效果，证明轻量级提示工程在生物医学NLP中的潜力。
摘要：
生物医学文本中的共指消解因复杂的领域特定术语、指称形式的高度歧义性以及共指表达之间的长距离依赖而面临独特挑战。本研究针对生成式大语言模型（LLMs）在生物医学领域的共指消解任务开展了全面评估。我们以CRAFT语料库为基准，通过四项提示工程实验评估LLMs的性能，这些实验在局部上下文增强、缩写与实体词典等领域特定线索的使用上各有不同。我们将这些方法与基于跨度的判别式编码器SpanBERT进行基准对比，以比较生成式与判别式方法的效能。结果表明，尽管LLMs展现出较强的表层共指处理能力，尤其是在辅以领域基础提示时，但其性能仍对长距离上下文和指称歧义较为敏感。值得注意的是，LLaMA 8B和17B模型在实体增强提示下表现出更优的精确率和F1值，凸显了轻量级提示工程在提升LLM用于生物医学自然语言处理任务效用方面的潜力。
英文摘要：
Coreference resolution in biomedical texts presents unique challenges due to complex domain-specific terminology, high ambiguity in mention forms, and long-distance dependencies between coreferring expressions. In this work, we present a comprehensive evaluation of generative large language models (LLMs) for coreference resolution in the biomedical domain. Using the CRAFT corpus as our benchmark, we assess the LLMs' performance with four prompting experiments that vary in their use of local, contextual enrichment, and domain-specific cues such as abbreviations and entity dictionaries. We benchmark these approaches against a discriminative span-based encoder, SpanBERT, to compare the efficacy of generative versus discriminative methods. Our results demonstrate that while LLMs exhibit strong surface-level coreference capabilities, especially when supplemented with domain-grounding prompts, their performance remains sensitive to long-range context and mentions ambiguity. Notably, the LLaMA 8B and 17B models show superior precision and F1 scores under entity-augmented prompting, highlighting the potential of lightweight prompt engineering for enhancing LLM utility in biomedical NLP tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
