当用词引导评估：在LLM评估中框架偏见的影响
When Wording Steers the Evaluation: Framing Bias in LLM judges
AI总结：
研究揭示了LLM评估中框架偏见的影响，通过设计对称提示展示框架如何扭曲模型判断，强调需建立框架意识的评估协议。
摘要：
大型语言模型（LLMs）因其响应会因提示语的措辞而变化而闻名，表明细微的引导可以影响其回答。然而，这种框架偏见对基于LLM的评估的影响仍鲜为人知，因为在评估中模型被期望做出稳定和公正的判断。受心理学中的框架效应启发，我们系统地研究了有意提示框架如何在四个高风险评估任务中扭曲模型判断。我们设计了对称提示，使用谓语正负构造，并展示了这种框架会引发模型输出的显著差异。在14个LLM评估者中，我们观察到对框架的明显易感性，模型家族表现出不同的倾向，倾向于同意或拒绝。这些发现表明，框架偏见是当前基于LLM的评估系统的一种结构性属性，强调了需要框架意识的协议。
英文摘要：
Large language models (LLMs) are known to produce varying responses depending on prompt phrasing, indicating that subtle guidance in phrasing can steer their answers. However, the impact of this framing bias on LLM-based evaluation, where models are expected to make stable and impartial judgments, remains largely underexplored. Drawing inspiration from the framing effect in psychology, we systematically investigate how deliberate prompt framing skews model judgments across four high-stakes evaluation tasks. We design symmetric prompts using predicate-positive and predicate-negative constructions and demonstrate that such framing induces significant discrepancies in model outputs. Across 14 LLM judges, we observe clear susceptibility to framing, with model families showing distinct tendencies toward agreement or rejection. These findings suggest that framing bias is a structural property of current LLM-based evaluation systems, underscoring the need for framing-aware protocols.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
