整合式解码：通过隐式自一致性提升事实性
Integrative Decoding: Improve Factuality via Implicit Self-consistency
AI总结：
本文提出整合式解码（ID），通过隐式自一致性在开放式生成中提升大模型事实性，在多个基准上显著改进并随采样扩展。
摘要：
基于自一致性的方法，即重复采样多个输出并选择最一致的一个作为最终响应，已被证明在提升大型语言模型的事实准确性方面非常有效。然而，现有方法通常对任务格式有严格限制，大大限制了其适用性。在本文中，我们提出了整合式解码（Integrative Decoding，ID），以释放自一致性在开放式生成任务中的潜力。ID 通过构建一组输入来运作，每个输入前都附加一个先前采样的响应，然后并发处理它们，在每一步解码时通过聚合所有对应预测来选择下一个词元。本质上，这种简单的方法隐式地将自一致性纳入了解码目标。广泛评估表明，ID 在多种语言模型上持续提升事实性，在 TruthfulQA（+11.2%）、Biographies（+15.4%）和 LongFact（+8.5%）基准上取得了显著改进。性能增益随着采样响应数量的增加而逐步放大，表明 ID 具有随重复采样扩展的潜力。
英文摘要：
Self-consistency-based approaches, which involve repeatedly sampling multiple outputs and selecting the most consistent one as the final response, prove to be remarkably effective in improving the factual accuracy of large language models. Nonetheless, existing methods usually have strict constraints on the task format, largely limiting their applicability. In this paper, we present Integrative Decoding (ID), to unlock the potential of self-consistency in open-ended generation tasks. ID operates by constructing a set of inputs, each prepended with a previously sampled response, and then processes them concurrently, with the next token being selected by aggregating of all their corresponding predictions at each decoding step. In essence, this simple approach implicitly incorporates self-consistency in the decoding objective. Extensive evaluation shows that ID consistently enhances factuality over a wide range of language models, with substantial improvements on the TruthfulQA (+11.2%), Biographies (+15.4%) and LongFact (+8.5%) benchmarks. The performance gains amplify progressively as the number of sampled responses increases, indicating the potential of ID to scale up with repeated sampling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
