我真的知道吗？学习事实自我验证以减少幻觉
Do I Really Know? Learning Factual Self-Verification for Hallucination Reduction
AI总结：
VeriFY通过训练时的自我验证框架减少LLM的事实性幻觉，通过一致性推理和阶段级损失屏蔽提升准确性与召回率。
摘要：
事实性幻觉仍然是大型语言模型（LLMs）面临的重大挑战。现有的缓解方法主要依赖于外部事后验证或在微调过程中直接将不确定性映射到回避，往往导致过于保守的行为。我们提出了VeriFY，一个训练时框架，该框架教导LLMs通过基于一致性的自我验证来推理事实不确定性。VeriFY通过结构化的验证轨迹增强训练，引导模型生成初始答案，生成并回答探测性验证查询，发出一致性判断，然后决定是否回答或回避。为了解决在增强轨迹上训练时可能强化幻觉内容的风险，我们引入了阶段级损失屏蔽方法，该方法排除了幻觉答案阶段的训练目标，同时保持对验证行为的监督。在多个模型家族和规模上，VeriFY将事实性幻觉率降低了9.7至53.3个百分点，召回率的减少仅在0.4至5.7个百分点之间，且在训练于单一来源时能够跨数据集泛化。源代码、训练数据和训练好的模型检查点将在接受后发布。
英文摘要：
Factual hallucination remains a central challenge for large language models (LLMs). Existing mitigation approaches primarily rely on either external post-hoc verification or mapping uncertainty directly to abstention during fine-tuning, often resulting in overly conservative behavior. We propose VeriFY, a training-time framework that teaches LLMs to reason about factual uncertainty through consistency-based self-verification. VeriFY augments training with structured verification traces that guide the model to produce an initial answer, generate and answer a probing verification query, issue a consistency judgment, and then decide whether to answer or abstain. To address the risk of reinforcing hallucinated content when training on augmented traces, we introduce a stage-level loss masking approach that excludes hallucinated answer stages from the training objective while preserving supervision over verification behavior. Across multiple model families and scales, VeriFY reduces factual hallucination rates by 9.7 to 53.3 percent, with only modest reductions in recall (0.4 to 5.7 percent), and generalizes across datasets when trained on a single source. The source code, training data, and trained model checkpoints will be released upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
