面向细粒度信息抽取的带增强指令的大语言模型基准测试
Benchmarking Large Language Models with Augmented Instructions for Fine-grained Information Extraction
AI总结：
本研究针对大语言模型适配细粒度信息抽取的需求，构建了带增强指令的细粒度IE基准数据集，实验发现模型性能不由规模单一决定，架构等因素也很关键，为LLM在IE领域的精细化应用提供参考。
摘要：
信息抽取（IE）是自然语言处理中的一项核心任务。传统方法依赖于使用简单指令的粗粒度抽取。然而，随着大语言模型（LLMs）的出现，有必要调整信息抽取技术以充分利用这些模型的能力。本文提出了一个专为大语言模型打造的细粒度信息抽取基准数据集，该数据集为每种信息类型采用了增强指令，内容包含任务描述、抽取规则、输出格式和示例。通过大量评估，我们发现编码器-解码器模型（尤其是T5和FLAN-T5）在泛化到未见过的信息类型方面表现优异，而ChatGPT对新任务形式展现出更强的适应能力。我们的结果还表明，模型性能并非仅由模型规模决定，同时凸显了架构、数据多样性和学习技术的重要性。这项工作为大语言模型在信息抽取领域更精细、更通用的应用铺平了道路。
英文摘要：
Information Extraction (IE) is an essential task in Natural Language Processing. Traditional methods have relied on coarse-grained extraction with simple instructions. However, with the emergence of Large Language Models (LLMs), there is a need to adapt IE techniques to leverage the capabilities of these models. This paper introduces a fine-grained IE benchmark dataset tailored for LLMs, employing augmented instructions for each information type, which includes task descriptions, extraction rules, output formats, and examples. Through extensive evaluations, we observe that encoder-decoder models, particularly T5 and FLAN-T5, perform well in generalizing to unseen information types, while ChatGPT exhibits greater adaptability to new task forms. Our results also indicate that performance is not solely dictated by model scale, and highlight the significance of architecture, data diversity, and learning techniques. This work paves the way for a more refined and versatile utilization of LLMs in Information Extraction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
