FactCheckmate:在语言模型中抢先检测并缓解幻觉
FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMs
AI总结:
本文提出FactCheckmate框架,通过轻量级分类器基于语言模型隐藏状态在解码前抢先检测幻觉,并调整隐藏状态进行干预,在多模型和多数据集上实现了超70%的检测准确率及34.4%的事实性提升。
摘要:
语言模型(LMs)会产生幻觉。我们提出疑问:能否在幻觉发生之前对其进行检测和缓解?这项工作给出了肯定的回答,证明了LMs的内部表示提供了可用于此目的的丰富信号。我们引入了FactCheckmate,它通过学习一个分类器来抢先检测幻觉,该分类器基于模型在输入上产生的隐藏状态,在解码开始前预测LM是否会产生幻觉。如果检测到幻觉,FactCheckmate会通过调整LM的隐藏状态进行干预,使模型产生更具事实性的输出。FactCheckmate提供了新的见解,即LMs的内部工作机制可以通过其隐藏状态揭示。在实践中,其检测和缓解模型均为轻量级,几乎不增加推理开销;与许多事后替代方案相比,FactCheckmate被证明是一种更高效的缓解幻觉的方法。我们在不同规模和模型家族(包括Llama、Mistral、Qwen和Gemma)的LMs上,跨不同领域的多种QA数据集评估了FactCheckmate。我们的结果证明了FactCheckmate的有效性,实现了超过70%的抢先检测准确率。平均而言,经过干预的LM生成的输出比未干预的输出事实性提高了34.4%。
英文摘要:
Language models (LMs) hallucinate. We inquire: Can we detect and mitigate hallucinations before they happen? This work answers this research question in the positive, by showing that the internal representations of LMs provide rich signals that can be used for this purpose. We introduce FactCheckmate, which preemptively detects hallucinations by learning a classifier that predicts whether the LM will hallucinate, based on the model's hidden states produced over the inputs, before decoding begins. If a hallucination is detected, FactCheckmate then intervenes by adjusting the LM's hidden states such that the model will produce more factual outputs. FactCheckmate provides fresh insights that the inner workings of LMs can be revealed by their hidden states. Practically, both its detection and mitigation models are lightweight, adding little inference overhead; FactCheckmate proves a more efficient approach for mitigating hallucinations compared to many post-hoc alternatives. We evaluate FactCheckmate over LMs of different scales and model families (including Llama, Mistral, Qwen and Gemma), across a variety of QA datasets from different domains. Our results demonstrate the effectiveness of FactCheckmate, achieving over 70% preemptive detection accuracy. On average, outputs generated by LMs with intervention are 34.4% more factual compared to those without.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。