CitaLaw：增强法律领域大语言模型的引用能力
CitaLaw: Enhancing LLM with Citations in Legal Domain
AI总结：
提出首个法律引用基准CitaLaw，通过参考语料库检索与三段论评估，增强LLM生成合法且带引用的回答，实验证明其有效性。
摘要：
在本文中，我们提出了CitaLaw，这是第一个旨在评估大语言模型（LLM）生成具有适当引用的合法合理回答能力的基准。CitaLaw包含了面向普通人和从业者的多样化法律问题，并配有一个包含法律条文和判例的综合语料库作为参考池。该框架使基于LLM的系统能够从参考语料库中检索支持性引用，并将这些引用与回答中的相应句子对齐。此外，我们引入了基于三段论的评估方法，以评估检索到的引用与LLM生成的回答之间的法律一致性，以及它们与用户问题的一致性。对2个开放领域和7个法律特定LLM的大量实验表明，整合法律引用显著提高了回答质量。此外，我们提出的基于三段论的评估方法与人类判断表现出高度一致性。
英文摘要：
In this paper, we propose CitaLaw, the first benchmark designed to evaluate LLMs' ability to produce legally sound responses with appropriate citations. CitaLaw features a diverse set of legal questions for both laypersons and practitioners, paired with a comprehensive corpus of law articles and precedent cases as a reference pool. This framework enables LLM-based systems to retrieve supporting citations from the reference corpus and align these citations with the corresponding sentences in their responses. Moreover, we introduce syllogism-inspired evaluation methods to assess the legal alignment between retrieved references and LLM-generated responses, as well as their consistency with user questions. Extensive experiments on 2 open-domain and 7 legal-specific LLMs demonstrate that integrating legal references substantially enhances response quality. Furthermore, our proposed syllogism-based evaluation method exhibits strong agreement with human judgments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
