面向复杂信息抽取任务的问答模型微调与对齐
Fine-tuning and aligning question answering models for complex information extraction tasks
AI总结：
本文提出整合并微调德语抽取式 QA 模型，以少量标注数据提升复杂业务文档信息抽取性能，并结合多种指标构建贴近人工评估的组合度量。
摘要：
大型语言模型（LLM）的出现提升了多种 NLP 任务的性能与可能性。尽管 ChatGPT 等生成式 AI 模型的使用为若干业务用例带来了新机会，但它们当前倾向于幻觉出虚假内容，这严重限制了其在文档分析（例如从文档中进行信息检索）中的适用性。相比之下，问答（QA）或段落检索等抽取式语言模型保证查询结果可在相应上下文文档的边界内找到，这使其成为企业生产环境中更可靠信息抽取的候选方案。在本工作中，我们提出一种方法，使用并整合抽取式 QA 模型，将德语业务文档（如保险报告或医疗说明书）的特征抽取改进到文档分析解决方案中。我们进一步表明，即使只使用少量标注数据，微调现有德语 QA 模型也能提升针对复杂语言特征（如损害原因解释或药物外观描述）的定制化抽取任务的性能。最后，我们讨论了评分指标在评估信息抽取任务中的相关性，并从 Levenshtein 距离、F1-Score、Exact Match 和 ROUGE-L 推导出一个组合指标，以模拟人类专家的评估标准。
英文摘要：
The emergence of Large Language Models (LLMs) has boosted performance and possibilities in various NLP tasks. While the usage of generative AI models like ChatGPT opens up new opportunities for several business use cases, their current tendency to hallucinate fake content strongly limits their applicability to document analysis, such as information retrieval from documents. In contrast, extractive language models like question answering (QA) or passage retrieval models guarantee query results to be found within the boundaries of an according context document, which makes them candidates for more reliable information extraction in productive environments of companies. In this work we propose an approach that uses and integrates extractive QA models for improved feature extraction of German business documents such as insurance reports or medical leaflets into a document analysis solution. We further show that fine-tuning existing German QA models boosts performance for tailored extraction tasks of complex linguistic features like damage cause explanations or descriptions of medication appearance, even with using only a small set of annotated data. Finally, we discuss the relevance of scoring metrics for evaluating information extraction tasks and deduce a combined metric from Levenshtein distance, F1-Score, Exact Match and ROUGE-L to mimic the assessment criteria from human experts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
