评估RAG系统中的社会偏见：当外部上下文有助于而推理有害
Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts
AI总结：
本研究评估RAG系统中的社会偏见，发现外部上下文可减少偏见，但链式思维提示反而增加偏见，凸显了需要偏见意识推理框架的重要性。
摘要：
大型语言模型（LLMs）中固有的社会偏见引发了重大公平性问题。检索增强生成（RAG）架构，通过检索外部知识源来增强LLMs的生成能力，仍然面临相同的偏见相关挑战。本研究聚焦于评估和理解RAG的社会偏见影响。通过在各种检索语料库、LLMs和偏见评估数据集上进行广泛实验，涵盖超过13种不同的偏见类型，我们意外地发现RAG中偏见有所减少。这表明，引入外部上下文有助于对抗由刻板印象驱动的预测，可能通过多样化模型输出的上下文基础来提高公平性。为了更好地理解这一现象，我们通过将链式思维（CoT）提示整合到RAG中，并评估模型的CoT忠实性来探索模型的推理过程。我们的实验发现，随着从检索文档中纳入更多上下文信息，模型的偏见倾向在刻板印象反应和反刻板印象反应之间转换。有趣的是，我们发现虽然CoT提高了准确性，但与RAG中观察到的偏见减少相反，它在各数据集上增加了总体偏见，突显了需要能够缓解这种权衡的偏见意识推理框架的必要性。
英文摘要：
Social biases inherent in large language models (LLMs) raise significant fairness concerns. Retrieval-Augmented Generation (RAG) architectures, which retrieve external knowledge sources to enhance the generative capabilities of LLMs, remain susceptible to the same bias-related challenges. This work focuses on evaluating and understanding the social bias implications of RAG. Through extensive experiments across various retrieval corpora, LLMs, and bias evaluation datasets, encompassing more than 13 different bias types, we surprisingly observe a reduction in bias in RAG. This suggests that the inclusion of external context can help counteract stereotype-driven predictions, potentially improving fairness by diversifying the contextual grounding of the model's outputs. To better understand this phenomenon, we then explore the model's reasoning process by integrating Chain-of-Thought (CoT) prompting into RAG while assessing the faithfulness of the model's CoT. Our experiments reveal that the model's bias inclinations shift between stereotype and anti-stereotype responses as more contextual information is incorporated from the retrieved documents. Interestingly, we find that while CoT enhances accuracy, contrary to the bias reduction observed with RAG, it increases overall bias across datasets, highlighting the need for bias-aware reasoning frameworks that can mitigate this trade-off.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
