arXiv 2409.002092024-08-30cs.CLcs.LG

通过基于语义因果图的指令微调增强大语言模型的事件推理能力

Enhancing Event Reasoning in Large Language Models through Instruction Fine-Tuning with Semantic Causal Graphs

University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；Peraton Labs（Peraton实验室）

Mazal Bethany, Emet Bethany, Brandon Wherry, Cho-Yu Chiang, Nishant Vishwamitra, Anthony Rios, Peyman Najafirad

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出基于语义因果图的指令微调方法，通过SCG指令和LoRA增强LLM的事件检测能力，在多个事件检测指标上显著超越标准微调并优于GPT-4，同时保持通用推理能力。

摘要：

事件检测与文本推理已成为多个领域中的关键应用。尽管大语言模型（LLMs）近期在推理能力方面取得了令人瞩目的进展，但它们在事件检测上仍常常表现不佳，尤其是由于缺乏考虑事件触发词与事件类型之间因果关系的训练方法。为应对这一挑战，我们提出了一种用于事件检测的LLM指令微调新方法。我们的方法引入语义因果图（Semantic Causal Graphs，SCGs）来同时捕获文本中的因果关系和上下文信息。在SCGs的基础上，我们提出了SCG指令（SCG Instructions），通过聚焦事件触发词及其与事件类型的关系来微调LLM，并采用低秩适配（Low-Rank Adaptation，LoRA）以帮助保持LLM的通用推理能力。我们的评估表明，使用SCG指令训练LLM在事件触发词分类（Event Trigger Classification）上比标准指令微调平均高出35.69%。值得注意的是，我们微调后的Mistral 7B模型在关键事件检测指标上还超越了GPT-4，在事件触发词识别（Event Trigger Identification）上平均高出31.01%，在事件触发词分类上平均高出37.40%，在事件分类（Event Classification）上平均高出16.43%。我们分析了通用能力的保持情况，观察到在六个基准上平均仅下降2.03分。这项综合性研究在多个数据集、提示策略和训练方法上考察了多个LLM在事件检测任务中的表现。

英文摘要：

Event detection and text reasoning have become critical applications across various domains. While LLMs have recently demonstrated impressive progress in reasoning abilities, they often struggle with event detection, particularly due to the absence of training methods that consider causal relationships between event triggers and types. To address this challenge, we propose a novel approach for instruction fine-tuning LLMs for event detection. Our method introduces Semantic Causal Graphs (SCGs) to capture both causal relationships and contextual information within text. Building off of SCGs, we propose SCG Instructions for fine-tuning LLMs by focusing on event triggers and their relationships to event types, and employ Low-Rank Adaptation (LoRA) to help preserve the general reasoning abilities of LLMs. Our evaluations demonstrate that training LLMs with SCG Instructions outperforms standard instruction fine-tuning by an average of 35.69\% on Event Trigger Classification. Notably, our fine-tuned Mistral 7B model also outperforms GPT-4 on key event detection metrics by an average of 31.01\% on Event Trigger Identification, 37.40\% on Event Trigger Classification, and 16.43\% on Event Classification. We analyze the retention of general capabilities, observing only a minimal average drop of 2.03 points across six benchmarks. This comprehensive study investigates multiple LLMs for the event detection task across various datasets, prompting strategies, and training approaches.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。