探索社会偏见对大型语言模型中忠实性幻觉的因果效应
Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Models
AI总结：
本研究利用结构因果模型和偏见干预数据集，首次探究并验证了社会偏见是导致大型语言模型产生忠实性幻觉的重要因果因素。
摘要：
大型语言模型（LLMs）在各种任务中取得了显著成功，但仍然容易产生忠实性幻觉，即输出与输入不一致。在本研究中，我们调查社会偏见是否导致了这些幻觉，这一因果关系尚未被探索。一个关键挑战是控制上下文中的混淆因素，这使偏见状态与幻觉之间因果关系的隔离变得复杂。为此，我们利用结构因果模型（SCM）来建立并验证因果关系，并设计偏见干预以控制混淆因素。此外，我们开发了偏见干预数据集（BID），包含各种社会偏见，能够精确测量因果效应。在主流LLMs上的实验表明，偏见是导致忠实性幻觉的重要原因，且各偏见状态的影响方向不同。我们进一步分析了这些因果效应在不同模型中的范围，特别关注社会偏见主要针对的不公平幻觉，揭示了偏见对幻觉生成的微妙但显著的因果效应。
英文摘要：
Large language models (LLMs) have achieved remarkable success in various tasks, yet they remain vulnerable to faithfulness hallucinations, where the output does not align with the input. In this study, we investigate whether social bias contributes to these hallucinations, a causal relationship that has not been explored. A key challenge is controlling confounders within the context, which complicates the isolation of causality between bias states and hallucinations. To address this, we utilize the Structural Causal Model (SCM) to establish and validate the causality and design bias interventions to control confounders. In addition, we develop the Bias Intervention Dataset (BID), which includes various social biases, enabling precise measurement of causal effects. Experiments on mainstream LLMs reveal that biases are significant causes of faithfulness hallucinations, and the effect of each bias state differs in direction. We further analyze the scope of these causal effects across various models, specifically focusing on unfairness hallucinations, which are primarily targeted by social bias, revealing the subtle yet significant causal effect of bias on hallucination generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
