为提升法律问答中的事实性而微调大语言模型
Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering
AI总结：
针对法律问答领域大语言模型的幻觉问题，研究团队推出LegalHalBench基准及相关评估指标，提出结合行为克隆和HIPO的幻觉缓解方法，经真实数据实验验证，在多项新旧指标上均实现显著提升。
摘要：
幻觉（即生成错误或捏造的信息）仍是大语言模型（LLM）面临的一项关键挑战，在法律问答（QA）这类高风险领域中尤为突出。为降低法律问答中的幻觉率，我们首先推出了名为LegalHalBench的基准，以及三种用于评估LLM回答法律问题时常见幻觉的自动指标。随后，我们提出了一种幻觉缓解方法，该方法整合了行为克隆与一种新颖的难样本感知迭代直接偏好优化（HIPO）。我们开展了大量基于真实数据的实验，以验证所提方法的有效性。结果显示，该方法在多项指标上均取得了显著提升，包括新提出的无幻觉法条率、法条相关率、法律主张真实性，以及METEOR、BERTScore、ROUGE-L等传统指标和胜率。
英文摘要：
Hallucination, or the generation of incorrect or fabricated information, remains a critical challenge in large language models (LLMs), particularly in high-stake domains such as legal question answering (QA). In order to mitigate the hallucination rate in legal QA, we first introduce a benchmark called LegalHalBench and three automatic metrics to evaluate the common hallucinations when LLMs answer legal questions. We then propose a hallucination mitigation method that integrates behavior cloning and a novel Hard Sample-aware Iterative Direct Preference Optimization (HIPO). We conduct extensive real-data experiments to validate the effectiveness of our approach. Our results demonstrate remarkable improvements in various metrics, including the newly proposed Non-Hallucinated Statute Rate, Statute Relevance Rate, Legal Claim Truthfulness, as well as traditional metrics such as METEOR, BERTScore, ROUGE-L, and win rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
