DynHD: 通过去噪动态偏差学习检测扩散大语言模型的幻觉
DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning
AI总结：
本文提出DynHD，通过空间和时间视角解决扩散大语言模型幻觉检测中的信息密度不均和去噪动态建模问题，采用语义感知证据构造模块和偏差检测器提升检测性能。
摘要：
扩散大语言模型（D-LLMs）因其迭代细化能力成为替代自回归模型的有希望的替代方案。然而，幻觉仍然是阻碍其可靠性的关键问题。为检测模型输出中的幻觉响应，token级不确定性（例如熵）已被广泛用作指示潜在事实错误的有效信号。然而，D-LLMs的固定长度生成范式意味着tokens对幻觉检测的贡献不均，只有小部分提供有意义的信号。此外，去噪过程中不确定性演变趋势也可提供重要信号，强调了建模其去噪动态以进行幻觉检测的必要性。本文提出DynHD，从空间（token序列）和时间（去噪动态）视角弥合这些差距。为解决token间的信息密度不平衡，我们提出一个语义感知证据构造模块，通过过滤非信息性token并强调语义有意义的token来提取幻觉指示信号。为建模去噪动态以进行幻觉检测，我们引入一个参考证据生成器，学习不确定性证据的预期演变轨迹，以及一个基于偏差的幻觉检测器，通过测量观测轨迹与参考轨迹之间的差异进行预测。广泛实验表明，DynHD在多个基准和基础模型上均优于最先进的基线，同时实现了更高的效率。
英文摘要：
Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty, such as entropy, has been widely used to indicate potential factual errors. Nevertheless, unlike auto-regressive models that generate tokens sequentially, D-LLMs generate fixed-length sequences simultaneously, where only a small subset of tokens is informative for hallucination detection. Thus, aggregating uncertainty over all tokens can be suboptimal. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide valuable signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD, which bridges these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To handle the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by removing task-invariant structural tokens and emphasizing the uncertainty of the remaining informative tokens. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models. The code is available at: https://github.com/qyy11-com/DynHD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
