引用溯源：通过法律引用图检测和减少LLM引用幻觉
Citation Grounding Measures the Oracle: Graph Coverage Determines Reported LLM Hallucination Rates in Law
AI总结：
提出引用溯源（CG）指标，利用乌克兰法院判决的引用图（1.008亿判决，5.02亿边）检测LLM法律引用幻觉，并通过CG-DPO方法（基于真实判决构建偏好对）减少幻觉，在100个法律查询上CG为0.791-0.873，幻觉率13-21%。
摘要：
大型语言模型系统性地产生法律引用幻觉——编造法规引用、引用已废除条款、混淆司法管辖区——但目前尚无自动化方法可大规模测量或减少此行为。我们提出引用溯源（CG），该指标通过从1.008亿乌克兰法院判决（5.02亿条边，21,736个唯一法规节点）中提取的真实引用图来验证LLM生成的法律引用。CG分解为三个组成部分——引用精确性（引用的条款是否存在？）、引用相关性（是否上下文相关？）和引用时效性（在相关日期是否有效？）——从而实现对幻觉类型的差异化诊断。对100个乌克兰法律查询的实证评估（涉及五个系统：通过AWS Bedrock的四个商业LLM——Claude Haiku 4.5、Mistral Pixtral Large、Amazon Nova Pro/Lite——以及一个RAG增强的生产系统）显示CG范围为0.791至0.873，其中13-21%的引用是幻觉。为了在没有人工标注的情况下减少幻觉，我们引入了引用溯源DPO（CG-DPO）：一种通过四种针对性策略从真实法院判决中破坏已验证引用来自动构建偏好对的方法。在包含2,244个法院判决的数据集上，使用LoRA微调的Qwen2.5-7B-Instruct模型在区分正确和错误引用方面达到了98.5%的平均验证准确率（奖励边际+14.9，3个种子的标准差<0.3个百分点）。引用图、评估框架和CG-DPO数据集作为开放资源发布。
英文摘要：
Verifying LLM-generated legal citations against a graph of citations extracted from real court decisions is an appealing way to measure hallucination at scale: no annotators, no reference answers. We show that what such a metric reports is governed by the coverage of the graph it queries rather than by the model it evaluates, and that at the coverage where its verdicts become trustworthy it stops distinguishing models at all. We score 400 responses (100 Ukrainian legal queries x four commercial LLMs) against two snapshots of the same national citation graph, holding responses, extractor and metric fixed. Against a sparse snapshot (4.7e5 records) citation grounding ranges 0.791-0.855, apparently showing 15-21% of citations hallucinated. Against a dense snapshot of the same registry re-derived ten weeks later (3.3e8 records, 5.8e7 decisions) the identical responses score 0.989-0.999. Subsampling shows the cause is coverage: harvesting modelled as uniform record sampling, calibrated on nothing but the record count, reproduces the sparse scores to within 0.018 while knowing nothing about the models. Bootstrapping over the queries shows the other half, which no version of this work reported: no pair of systems is separable at 95% at any oracle size tested. The metric is caught between two failures. Sparse oracles discriminate, but what they discriminate is harvesting coverage; dense oracles are trustworthy and separate nothing. Densifying the graph, the obvious remedy for the first, produces the second. An independent legislation registry adjudicates: all 54 citations flagged by the sparse oracle name real articles, a 100% false-positive rate; of the four flagged by the dense oracle, two are fabrications, one a coverage gap, and one we cannot classify - it looks like a repealed provision, which is the argument in miniature.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
