基于知识图谱的LLM后训练以增强法律推理
Knowledge Graph-Assisted LLM Post-Training for Enhanced Legal Reasoning
AI总结：
本文提出基于知识图谱的LLM后训练方法，通过构建法律知识图谱提升法律推理能力，在多个基准测试中优于基线模型。
摘要：
LLM后训练主要依赖于大规模文本语料和人类反馈，没有捕捉到领域知识的结构。这导致模型在处理复杂推理任务时遇到困难，尤其是在高风险的专业领域。在法律领域，推理需要深入理解各种法律概念之间的关系，这是当前LLM后训练的关键缺失部分。在本文中，我们提出了一种基于知识图谱（KG）的方法，以增强LLM在法律领域的推理能力，并可推广到其他高风险领域。我们通过遵循IRAC（问题、规则、分析和结论）框架来建模关键法律概念，并构建包含12,000个法律案例的知识图谱。然后我们使用我们的IRAC知识图谱生成训练数据，并使用三种最先进的（SOTA）LLM（30B、49B和70B）进行监督微调（SFT）和直接偏好优化（DPO），并改变架构和基础模型家族。我们的后训练模型在4/5个不同的法律基准测试（14个任务）上比基线模型取得了更好的平均性能。特别是，我们的70B DPO模型在6个推理任务中的4个任务上，比基线和一个141B最先进的法律LLM取得了最佳分数，证明了我们的知识图谱在增强LLM法律推理能力方面的有效性。
英文摘要：
LLM post-training has primarily relied on large text corpora and human feedback, without capturing the structure of domain knowledge. This has caused models to struggle dealing with complex reasoning tasks, especially for high-stakes professional domains. In Law, reasoning requires deep understanding of the relations between various legal concepts, a key component missing in current LLM post-training. In this paper, we propose a knowledge graph (KG)-assisted approach for enhancing LLMs' reasoning capability in Legal that is generalizable to other high-stakes domains. We model key legal concepts by following the \textbf{IRAC} (Issue, Rule, Analysis and Conclusion) framework, and construct a KG with 12K legal cases. We then produce training data using our IRAC KG, and conduct both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with three state-of-the-art (SOTA) LLMs (30B, 49B and 70B), varying architecture and base model family. Our post-trained models obtained better average performance on 4/5 diverse legal benchmarks (14 tasks) than baselines. In particular, our 70B DPO model achieved the best score on 4/6 reasoning tasks, among baselines and a 141B SOTA legal LLM, demonstrating the effectiveness of our KG for enhancing LLMs' legal reasoning capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
