AMEL: 累积消息对LLM判断的影响
AMEL: Accumulated Message Effects on LLM Judgments
AI总结：
研究LLM在对话中因历史消息极性而偏离基准判断的累积消息效应（AMEL），发现模型偏向历史主流极性，且负向历史偏见更强，但偏见不随上下文长度增长，简单修复是为每个项目使用新上下文。
摘要：
大型语言模型常被用作自动评估者：审查代码、审核内容或评分输出，通常许多项目通过一次对话处理。我们询问先前对话历史的极性是否会偏倚后续判断，我们将这种效应称为LLM判断的累积消息效应（AMEL）。通过对来自4个提供商（OpenAI、Anthropic、Google和四个开源模型）的11个模型进行75,898次API调用，我们在隔离或跟随以正面或负面评价为主的历史之后呈现相同的测试项目。模型倾向于对话的主流极性（d = -0.17, p < 10^-46）。该效应集中在模型在基线时真正不确定的项目上（高熵项目d = -0.34，而基线确定时d = -0.15）。偏见不随上下文长度增长：5个先前轮次和50个产生相同的偏移（Spearman |r| < 0.01；OLS斜率p = 0.80）。并且存在负性不对称：按项目配对，负面历史诱导的偏见是正面的1.62倍（t = 13.46, p < 10^-39, n = 2,481）。扩展规模有帮助但不能解决（Anthropic: Haiku -0.22到Opus -0.17；OpenAI: Nano -0.34到GPT-5.2 -0.17）。三项后续研究缩小了机制范围。令牌概率分布连续变化，而非在阈值处。负性不对称既有令牌级成分也有语义成分，尽管在我们的样本量下平衡归因是探索性的。位置不重要：在50轮历史中任何位置的五个有偏轮次产生相同的偏移。评估流程最简单的修复是为每个项目使用新上下文；当批处理不可避免时，平衡历史有帮助。
英文摘要：
Large language models are routinely used as automated evaluators: to review code, moderate content, or score outputs, often with many items passing through one conversation. We ask whether the polarity of prior conversation history biases subsequent judgments, an effect we call the accumulated message effect on LLM judgments (AMEL). Across 84,088 API calls to 12 models from 5 providers (OpenAI, Anthropic, Google, DeepSeek, and four open-source models), we present identical test items in isolation or following histories saturated with predominantly positive or negative evaluations. Models shift toward the conversation's prevailing polarity (d = -0.17, p < 10^-53). The effect concentrates on items where the model is genuinely uncertain at baseline (d = -0.36 for high-entropy items, vs d = -0.15 when the baseline is deterministic). Bias does not grow with context length: 5 prior turns and 50 produce the same shift (Spearman |r| < 0.01; OLS slope p = 0.80). And there is a negativity asymmetry: paired per item, negative histories induce 1.52x more bias than positive (t = 13.03, p < 10^-36, n = 2,733). Scaling helps but does not solve it (Anthropic: Haiku -0.22 to Opus -0.17; OpenAI: Nano -0.34 to GPT-5.2 -0.17). Three follow-ups narrow the mechanism. The token probability distribution shifts continuously, not at a threshold. The negativity asymmetry has both token-level and semantic components, though attributing the balance is exploratory at our sample sizes. Position does not matter: five biased turns anywhere in a 50-turn history produce the same shift. The simplest fix for evaluation pipelines is a fresh context per item; when batching is unavoidable, balancing the history helps.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
