通过幻觉拒绝采样构建可靠的长文本生成
Building Reliable Long-Form Generation via Hallucination Rejection Sampling
AI总结：
提出分段幻觉拒绝采样框架SHARS，利用任意幻觉检测器在生成过程中拒绝并重采样幻觉片段，以缓解长文本生成中的幻觉累积问题，提升事实一致性。
摘要：
大型语言模型（LLMs）在开放式文本生成方面取得了显著进展，但仍容易产生不正确或无依据的幻觉内容，这损害了其可靠性。在长文本生成中，由于幻觉雪崩现象（早期错误传播并累积到后续输出），这一问题更加严重。为了解决这一挑战，我们提出了一种新颖的推理时幻觉缓解框架，称为分段幻觉拒绝采样（SHARS），该框架使用任意幻觉检测器在生成过程中识别并拒绝幻觉片段，并重新采样直到生成忠实的内容。通过仅保留可信信息并在此基础上构建后续生成，该框架减轻了幻觉累积并增强了事实一致性。为了实例化该框架，我们采用语义不确定性作为检测器，并引入了若干关键修改以解决其局限性并更好地适应长文本。我们的方法使模型能够自我纠正幻觉，无需外部资源（如网络搜索或知识库），同时保持与这些资源的兼容性以便未来扩展。在标准化幻觉基准上的实证评估表明，我们的方法显著减少了长文本生成中的幻觉，同时保持甚至提高了生成的信息量。代码可在以下网址获取：this https URL。
英文摘要：
Large language models (LLMs) have achieved remarkable progress in open-ended text generation, yet they remain prone to hallucinating incorrect or unsupported content, which undermines their reliability. This issue is exacerbated in long-form generation due to hallucination snowballing, a phenomenon where early errors propagate and compound into subsequent outputs. To address this challenge, we propose a novel inference-time hallucination mitigation framework, named Segment-wise HAllucination Rejection Sampling (SHARS), which uses an arbitrary hallucination detector to identify and reject hallucinated segments during generation and resample until faithful content is produced. By retaining only confident information and building subsequent generations upon it, the framework mitigates hallucination accumulation and enhances factual consistency. To instantiate this framework, we adopt semantic uncertainty as the detector and introduce several vital modifications to address its limitations and better adapt it to long-form text. Our method enables models to self-correct hallucinations without requiring external resources such as web search or knowledge bases, while remaining compatible with them for future extensions. Empirical evaluations on standardized hallucination benchmarks demonstrate that our method substantially reduces hallucinations in long-form generation while preserving or even improving the informativeness of generation. Code is available at: https://github.com/TreeLLi/hallucination-rejection-sampling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
