发表机构
Google（谷歌）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过平衡实验评估Gemma 4模型，发现来源框架比阅读顺序影响更大，且首因效应受表面措辞和结构重复显著调节。
AI 中文摘要
当语言模型接收到两份相互冲突的文档作为输入时，它如何决定优先考虑哪一份？它是依赖于来源的呈现方式（框架）还是文档的呈现顺序？我们在 Google 预训练的 Gemma 4-e4b 模型上，通过一个针对性的行为测试套件（n = 13 个项目，在短的单轮上下文中进行了 784 次前向传播），采用完全平衡的实验设计评估了这种行为。这种设置使我们能够在数学上分离来源框架和阅读位置的具体影响，同时确保模型自然的词汇偏差被抵消。在十个测试条件下，我们发现了以下结果：1. 来源框架在很大程度上压倒了阅读位置。当直接竞争时，来源的语义框架（例如将其呈现为官方指南或最新更新）对模型最终答案的影响显著强于文档的呈现顺序。2. 模型偏爱它首先阅读的文档，但这种偏差高度可变。虽然模型始终表现出首因效应（偏好首先呈现的文档），但这种偏差的实际强度仅基于表面措辞就波动了至少 5 倍。3. 整体结构重复，而非短复制线索，驱动位置偏差。模型对第一份文档的偏好并非对短重复触发短语（如“是 [答案]”）的机械反应。然而，当两份竞争文档在结构上完全相同，使用逐字逐句的模板时，首因效应确实显著增加。在两个来源之间引入整体措辞的变化会减少这种位置偏差。
英文摘要
When a language model receives two conflicting documents as input, how does it decide which one to prioritize? Does it rely on how the sources are framed or the presentation order of the documents? We evaluated this behavior on Google's pre-trained Gemma 4-e4b model across a targeted behavioral suite (n = 13 items, 784 forward passes in short, single-turn contexts) using a completely counterbalanced experimental design. This setup allowed us to mathematically isolate the specific effects of source framing and reading position, while ensuring the model's natural vocabulary biases were canceled out. Across ten test conditions, we discovered the following: 1. Source framing heavily overpowers reading position. When directly competing, the semantic framing of a source (such as presenting it as an official guideline or a fresh update) had a significantly stronger impact on the model's final answer than the presentation order of the document. 2. The model favors the first document it reads, but this bias is highly variable. While the model consistently demonstrated a primacy effect (preferring the first document presented), the actual strength of this bias fluctuated by at least a factor of 5 based solely on the surface wording. 3. Overall structural repetition, not short copy-cues, drives positional bias. The model's preference for the first document is not a mechanical reaction to short, repetitive trigger phrases, such as "is [Answer]". However, the primacy effect does increase significantly when the two competing documents are structurally identical, using word-for-word verbatim templates. Introducing variation in the overall wording between the two sources reduces this positional bias.
Comments36 pages, 1 figure, evaluation dataset and logs released
