OSCAR:协调自我验证与跨路径细化
OSCAR: Orchestrated Self-verification and Cross-path Refinement
AI总结:
OSCAR通过利用扩散语言模型的去噪轨迹,提出一种无需训练的推理框架,通过交叉链熵检测高不确定性位置并进行针对性修正,提升生成质量与事实准确性。
摘要:
扩散语言模型(DLMs)暴露其去噪轨迹,为推理时控制提供了自然手段;因此,理想的幻觉缓解框架应在生成过程中利用此模型原生信号干预,而非依赖外部训练的幻觉分类器。为此,我们提出了承诺不确定性定位:给定去噪轨迹,识别出在事实不可靠承诺传播前,交叉链熵超过无监督阈值的token位置。我们引入了一系列轨迹级评估,包括交叉链在幻觉处的分歧度(CDH)度量,用于原理性比较定位方法。我们还引入了OSCAR,一种无需训练的推理时框架,通过此公式操作化。OSCAR运行N个并行去噪链,使用随机揭示顺序,计算交叉链香农熵以检测高不确定性位置,然后根据检索到的证据进行针对性掩码。消融实验确认,定位和修正贡献互补增益,稳健性在N∈{4,8,16}中表现良好。在使用LLaDA-8B和Dream-7B的TriviaQA、HotpotQA、RAGTruth和CommonsenseQA上,OSCAR通过不确定性引导的掩码显著减少幻觉内容并提高事实准确性,同时促进更有效的检索证据整合。其基于原生熵的不确定性信号优于专门训练的检测器,突显了扩散语言模型内在识别事实不确定性的能力,这种能力在自回归模型的序列token承诺结构中不存在。
英文摘要:
Diffusion language models (DLMs) expose their denoising trajectories, offering a natural handle for inference-time control; accordingly, an ideal hallucination mitigation framework should intervene during generation using this model-native signal rather than relying on an externally trained hallucination classifier. Toward this, we formulate commitment uncertainty localization: given a denoising trajectory, identify token positions whose cross-chain entropy exceeds an unsupervised threshold before factually unreliable commitments propagate into self-consistent but incorrect outputs. We introduce a suite of trajectory-level assessments, including a cross-chain divergence-at-hallucination (CDH) metric, for principled comparison of localization methods. We also introduce OSCAR, a training-free inference-time framework operationalizing this formulation. OSCAR runs N parallel denoising chains with randomized reveal orders, computes cross-chain Shannon entropy to detect high-uncertainty positions, and then performs targeted remasking conditioned on retrieved evidence. Ablations confirm that localization and correction contribute complementary gains, robust across N in {4, 8, 16}. On TriviaQA, HotpotQA, RAGTruth, and CommonsenseQA using LLaDA-8B and Dream-7B, OSCAR enhances generation quality by significantly reducing hallucinated content and improving factual accuracy through uncertainty-guided remasking, which also facilitates more effective integration of retrieved evidence. Its native entropy-based uncertainty signal surpasses that of specialized trained detectors, highlighting an inherent capacity of diffusion language models to identify factual uncertainty that is not present in the sequential token commitment structure of autoregressive models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。