LLM能否比生成更好地判断？探究上下文问答中的任务不对称性、机制可解释性与可迁移性
Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA
AI总结：
本研究在受控上下文问答中测试LLM自我评估是否比生成更容易，发现评估并非普遍更易，注意力分析显示评估对上下文关注更少，微调实验证实不对称性非训练伪影，挑战了自我评估管线的核心假设。
摘要：
LLM-as-a-Judge和自我评估管线隐含假设评估比生成更容易。我们在受控的上下文问答设置中测试这一假设，其中上下文段落是唯一信息源，每个模型判断自己生成的答案，消除了开放域比较中的参数知识混淆。在四个基准（SQuAD 2.0、DROP、HotpotQA、MuSiQue）和两个模型上，评估并非普遍更容易：在四个基准中的三个上，生成准确率超过自我评估，多跳MuSiQue是例外。注意力分析揭示了原因：评估对上下文的关注比生成少3-5倍，且几乎不阅读候选答案。LoRA微调证实不对称性并非训练伪影：生成微调导致过度接受，评估微调降低生成性能。这些发现挑战了自我评估管线中的核心假设。
英文摘要：
LLM-as-a-Judge and self-evaluation pipelines implicitly assume that evaluation is easier than generation. We test this in a controlled in-context QA setting where a context passage is the sole information source and each model judges the answer it generated, removing the parametric-knowledge confound of open-domain comparisons. Across four benchmarks (SQuAD 2.0, DROP, HotpotQA, MuSiQue) and two models, evaluation is not uniformly easier: generation accuracy exceeds self-evaluation on three of four, with multi-hop MuSiQue the exception. Attention analysis reveals why: evaluation attends to context 3--5x less than generation does and barely reads the candidate answer. LoRA fine-tuning confirms the asymmetry is not a training artifact: generation fine-tuning induces over-acceptance and evaluation fine-tuning degrades generation. These findings challenge core assumptions in self-evaluation pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
