更细致的引用总是更好吗?重新思考属性生成中的粒度
Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation
AI总结:
研究探讨了属性生成中引用粒度对模型性能的影响,发现中间粒度(段落级)表现最佳,细粒度引用会破坏语义依赖,粗粒度则引入噪声,需优化粒度以提升属性质量和生成可靠性。
摘要:
引用粒度——是否引用单个句子、段落或文档——是属性生成中的关键设计选择。尽管细粒度引用常用于精确的人工验证,但其对模型性能的影响尚未充分研究。我们分析了四种模型规模(8B-120B),发现强制细粒度引用会使属性质量下降16-276% compared to最佳表现粒度。我们观察到属性质量在中间粒度(段落级)达到峰值。我们的分析表明,细粒度(句子级)引用会破坏将证据归因于回答声明所需的语义依赖,而过度粗粒度(多段落)则引入干扰噪声。重要的是,性能差距的大小与模型规模非单调变化:细粒度约束对大模型惩罚更大,表明原子引用单位会破坏这些模型擅长的多句信息合成。令人惊讶的是,最优引用粒度可显著提升属性质量,同时保持或改善回答正确性。总体而言,我们的发现表明,仅通过细粒度引用优化人工验证会忽视模型约束,影响属性忠实度和生成可靠性。相反,有效的属性生成需要将引用粒度与模型的自然语义范围对齐。
英文摘要:
Citation granularity -- whether to cite individual sentences, paragraphs, or documents -- is a critical design choice in attributed generation. While fine-grained citations are commonly preferred for precise human verification, their impact on model performance remains under-explored. We analyze four model scales (8B-120B) and demonstrate that enforcing fine-grained (sentence-level) citations forfeits gains of 2-97% (median 40%) relative to the best-performing granularity, and up to 338% on individual tasks. Strikingly, setting citation granularity to its optimal value (based on attribution quality) unlocks these substantial gains while leaving overall answer correctness essentially unchanged (between -2.3% and +4.4%). We observe a consistent pattern where attribution quality peaks at intermediate (paragraph-level) granularities: finer citations appear to sever the semantic dependencies needed to ground a claim, while excessively coarse citations introduce distracting noise. Importantly, this performance gap varies with scale: when a claim rests on a small or moderate amount of evidence, it disproportionately penalizes larger models by disrupting the multi-sentence information synthesis at which they excel. Fine-grained citation rests on the premise that a sentence is a sufficient unit of evidence on its own. Our results indicate that it often is not, and that this is a property of the model rather than of the citation standard. Standards fixed for human verifiability may therefore paradoxically degrade the very attribution they aim to ensure; effective attribution requires matching granularity to the model's semantic scope rather than fixing it by convention.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。