arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.09971cs.CL
收藏

构建用于对抗大语言模型幻觉的基准与干预方法

Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs

Google Research(谷歌研究院);Technion – Israel Institute of Technology(以色列理工学院)

Adi Simhi, Jonathan Herzig, Idan Szpektor, Yonatan Belinkov

AI总结:

针对大语言模型幻觉干预研究缺乏统一基准、成因区分不足的问题,提出WACK分类方法及双场景基准,探究不同干预设置的效果,发现注意力块干预更优、幻觉前表征向量效果更好,并提出更鲁棒的动态干预方法。

摘要:

大语言模型(LLMs)容易产生幻觉,这引发了学界检测和预防幻觉的广泛研究。近期工作尝试通过干预模型的生成过程来缓解幻觉,通常的做法是计算幻觉生成与基于事实生成的表征向量,从而引导模型的隐藏状态偏离幻觉状态。然而,现有研究普遍采用不同的实验设置,且未合理区分幻觉的不同可能成因,导致干预措施存在偏差。\n在本研究中,我们提出了一种基于模型先验知识对样本进行分类的方法,命名为WACK。我们构建了WACK基准,支持开卷问答和闭卷问答两种场景下的干预研究。借助该基准,我们对不同干预选择的效果开展了广泛研究,包括被干预的组件、干预频率以及干预强度。我们发现干预效果因干预组件的不同而存在差异:注意力块的干预表现良好,而残差流则会损害语言建模能力。我们还表明,使用幻觉发生前(而非发生后)收集的表征向量,能让干预取得更好的效果。最后,我们提出了一种新型动态干预方法,该方法仅在必要时进行干预,因此比标准静态干预更具鲁棒性。相关代码可在https://github.com/technion-cs-nlp/hallucination-mitigation 获取。

英文摘要:

Large language models (LLMs) are prone to hallucinations, which sparked a widespread effort to detect and prevent them. Recent work attempts to mitigate hallucinations by intervening in the model's generation, typically computing representative vectors of hallucinations vs. grounded generations, for steering the model's hidden states away from a hallucinatory state. However, common studies employ different setups and do not properly separate different possible causes of hallucinations, making interventions misguided. In this work, we introduce a method for categorizing examples based on the model's prior knowledge, named WACK. We construct WACK benchmarks that support interventions in two settings: open-book and closed-book question answering. Using the benchmarks, we perform an extensive investigation of the effect of different choices for intervention, such as the intervened components, and how often and how strongly to intervene. We find that intervention success varies depending on the component, with the attention blocks performing well and the residual stream proving detrimental to language modeling capabilities. We also show that interventions can benefit from representative vectors collected before, rather than after, a hallucination occurs. Finally, we introduce a new dynamic intervention, which intervenes only if needed, and thus is more robust than standard static interventions. The code is available at https://github.com/technion-cs-nlp/hallucination-mitigation .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑