科学文献中的引用归因：新基准与方法
Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations
AI总结：
针对LLM引用幻觉问题，提出REASONS基准（12,723个实例）和AR/HR双指标框架，评估多种归因方法，发现高级RAG降低幻觉率但减少弃权，强调系统需在不确定时适当弃权。
摘要：
大型语言模型（LLMs）越来越多地生成带有引用的回复，然而引用幻觉仍然是可信科学信息获取中的重大挑战。我们提出了REASONS，一个包含12,723个句子级引用实例、覆盖12个arXiv学科类别的基准，旨在评估不同证据条件下的科学引用归因。我们提出了一个由弃权率（AR）和幻觉率（HR）组成的双指标框架，以刻画可靠性与响应性之间的权衡。利用作者归因和标题归因任务，我们在零上下文、元数据增强、级联元数据增强提示（CMP）、检索增强和对抗性设置下评估了专有和开源LLMs。高级RAG相比朴素RAG降低了HR（65.4%对87.6%），但将AR从5.0%降至0%。在对抗性元数据下，多个系统的HR超过85%，而检索增强变体通常保持接近零的弃权。对1,000个输出的人工评估（κ=0.78）发现事实幻觉与可接受释义的比例为12.7:1。我们的研究结果表明，引用归因系统不仅应评估其正确性，还应评估其在不确定性下适当弃权（不执行）的能力。REASONS为研究引用生成中的归因可靠性提供了基准和评估框架。
英文摘要：
Large language models (LLMs) increasingly generate citation-backed responses, yet citation hallucination remains a major challenge for trustworthy scientific information access. We introduce REASONS, a benchmark of 12,723 sentence-level citation instances spanning 12 arXiv subject categories, designed to evaluate scientific citation attribution under varying evidence conditions. We propose a dual-metric framework consisting of Abstention Rate (AR) and Hallucination Rate (HR) to characterize the trade-off between reliability and responsiveness. Using author-attribution and title-attribution tasks, we evaluate proprietary and open-source LLMs under zero-context, metadata-augmented, cascaded metadata-augmented prompting (CMP), retrieval-augmented, and adversarial settings. Advanced RAG lowers HR relative to Naive RAG (65.4% vs. 87.6%) but reduces AR from 5.0% to 0%. Under adversarial metadata, several systems exceed 85% HR, while retrieval-augmented variants frequently maintain near-zero abstention. Human evaluation of 1,000 outputs ($κ=0.78$) finds a 12.7:1 ratio of factual hallucinations to acceptable paraphrases. Our findings demonstrate that citation attribution systems should be evaluated not only for correctness but also for their ability to abstain appropriately under uncertainty. REASONS provides a benchmark and evaluation framework for studying attribution reliability in citation generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
