温度如何塑造检索增强生成中的意识形态话语？

中文摘要 AI 辅助

检索增强生成（RAG）越来越多地被用于减少大语言模型（LLM）的幻觉并加强其事实基础。尽管已经探讨了对检索过程中错误的鲁棒性，但意识形态偏差对LLM输出的影响却被忽视。例如，若检索到的材料包含意识形态立场，RAG可能在其输出中传递、放大或抑制此类意识形态话语。在本研究中，我们通过检查包含意识形态话语的RAG框架在LLM生成答案中的影响来解决此问题。为此，我们对1117篇新冠治疗文章的语料库应用词汇多维分析（LMDA），识别出三种意识形态话语。该语料库随后用作RAG的外部知识源。我们通过让模型在不同采样温度下回答意识形态问题来评估多个LLM。根据生成文本与意识形态参考文本的相似性对其进行语义和词汇评估。我们的发现表明，RAG框架容易将意识形态话语转移到LLM响应中，采样温度对这种转移的强度有可测量的影响。生成答案与参考文本之间的话语对齐在中等温度下最高，此时模型在随机性与检索基础之间取得平衡，而在低温下下降，表明过度确定性采样会抑制话语转移。

英文摘要

Retrieval-Augmented Generation (RAG) has been increasingly adopted to reduce hallucinations and strengthen the factual grounding of large language models (LLMs). While robustness to errors in the retrieval process has been explored, the impact of ideological bias on LLM outputs has been overlooked. For instance, if the retrieved material contains ideological positions, the RAG may transmit, amplify, or suppress such ideological discourses in its outputs. In this study, we address this issue by examining the influence of the RAG framework, comprising ideological discourses, in LLM-generated answers. To this end, we applied Lexical Multidimensional Analysis (LMDA) on a corpus of 1,117 COVID-19 treatment articles, identifying three ideological discourses. This corpus is then used as the external knowledge source for the RAG. We assessed several LLMs by having the models answer ideological questions at different sampling temperatures. The generated texts were assessed semantically and lexically based on their similarities with ideological reference texts. Our findings show that the RAG framework is prone to transferring ideological discourses into LLM responses, with sampling temperature having a measurable impact on the strength of this transfer. Discoursive alignment between generated answers and the reference text is highest at moderate temperatures, where models balance stochasticity with retrieval grounding, and drops at low temperatures, indicating that overly deterministic sampling suppresses discourse transfer.

发表机构

• Wichita State University（威斯康星州立大学）
• São Paulo Catholic University（圣保罗天主教大学）

机构由 AI 辅助整理，请以论文原文为准。