发表机构
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics; The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（南京航空航天大学人工智能学院; 教育部脑机智能技术重点实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究基于不确定性的幻觉检测，提出面向多样性的微调策略，包括基于监督微调（SFT）和直接偏好优化（DPO）的方法，经实验验证可提高幻觉检测有效性，结果优于或可比现有方法。
AI 中文摘要
现有幻觉检测方法通常在推理阶段进行，不对模型本身做任何修改。本文旨在探索能增强最终模型中幻觉可检测性的微调策略。聚焦基于语义熵的检测，发现因模型多次运行产生近乎相同错误答案，许多错误输出未被检测到。为此提出面向多样性的微调以鼓励更多样化的生成。介绍了基于监督微调（SFT）和直接偏好优化（DPO）的两种具体策略。通过大量实验评估该方法并分析微调前后模型行为。发现采用微调方法后，模型对幻觉答案产生低语义熵响应的可能性降低，提高了幻觉检测有效性，结果优于或可比现有方法。代码将公开发布。
英文摘要
Existing hallucination detection methods are typically conducted at the inference stage, without making any modifications to the model itself. In this paper, we are interested in exploring fine-tuning strategies that enhance the detectability of hallucinations in the resulting model. Focusing on semantic-entropy-based detection, we observe that many erroneous outputs remain undetected because the model produces nearly identical incorrect answers across multiple runs. To address this, we propose diversity-oriented fine-tuning to encourage more varied generations. We introduce two specific strategies: one based on Supervised Fine-Tuning (SFT) and the other on Direct Preference Optimization (DPO). Extensive experiments are conducted to evaluate our approach and analyze the behavior of the models before and after fine-tuning. We find that after adopting our fine-tuning methods, the models become less likely to produce low semantic entropy responses for hallucinated answers, thereby improving the effectiveness of hallucination detection, eventually yielding results better than or comparable with state of the art methods. The code will be publicly released.
