知识遮蔽导致大语言模型中的融合式幻觉
Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models
AI总结：
本研究揭示大语言模型中由训练数据不平衡引发的“知识遮蔽”导致融合式幻觉的现象，验证其与泛化界的关联，并提出无训练自对比解码方法，实现高效幻觉预判与控制。
摘要：
幻觉通常被视为大语言模型（LLM）应用的一大障碍，尤其是在知识密集型任务中。即便训练语料完全由真实陈述构成，语言模型仍会以多事实融合的形式生成幻觉。我们将这一现象命名为“知识遮蔽”：当我们以多个条件向语言模型查询知识时，部分条件会遮蔽其他条件，进而产生幻觉输出。该现象部分源于训练数据不平衡，我们在多种语言模型家族和规模的预训练模型及微调模型上均验证了这一点。从理论角度看，知识遮蔽可被解释为对主导条件（模式）的过度泛化。我们发现，幻觉率会随（热门与冷门条件间的）不平衡比以及主导条件描述长度的增加而上升，这与我们推导的泛化界一致。最后，我们提出将遮蔽条件作为信号，在幻觉产生前将其捕捉，并提出一种无需训练的自对比解码方法，以在推理阶段缓解幻觉。我们提出的方法在不同模型和数据集上，幻觉预判的F1值最高可达82%，幻觉控制效果提升11.2%至39.4%。
英文摘要：
Hallucination is often regarded as a major impediment for using large language models (LLMs), especially for knowledge-intensive tasks. Even when the training corpus consists solely of true statements, language models still generate hallucinations in the form of amalgamations of multiple facts. We coin this phenomenon as ``knowledge overshadowing'': when we query knowledge from a language model with multiple conditions, some conditions overshadow others, leading to hallucinated outputs. This phenomenon partially stems from training data imbalance, which we verify on both pretrained models and fine-tuned models, over a wide range of LM model families and sizes.From a theoretical point of view, knowledge overshadowing can be interpreted as over-generalization of the dominant conditions (patterns). We show that the hallucination rate grows with both the imbalance ratio (between the popular and unpopular condition) and the length of dominant condition description, consistent with our derived generalization bound. Finally, we propose to utilize overshadowing conditions as a signal to catch hallucination before it is produced, along with a training-free self-contrastive decoding method to alleviate hallucination during inference. Our proposed approach showcases up to 82% F1 for hallucination anticipation and 11.2% to 39.4% hallucination control, with different models and datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
