利用判断分布改进LLM-as-a-Judge推理
Improving LLM-as-a-Judge Inference with the Judgment Distribution
AI总结：
本研究提出利用LLM法官输出的判断分布而非仅文本输出来改进推理，发现取均值优于贪婪解码，且风险规避方法有效，但思维链提示可能损害性能。
摘要：
使用语言模型来大规模近似人类对文本质量的偏好（LLM-as-a-judge）已成为适用于许多任务的标准实践。判断通常仅从法官的文本输出中提取，通常采用贪婪解码。然而，LLM法官自然提供判断令牌上的分布，这为提取细粒度偏好提供了多种推理方法。我们发现，在所有评估设置（即逐点、成对和列表式）中，取判断分布的均值始终优于取众数（即贪婪解码）。我们进一步探索了从判断分布中推导偏好的新方法，并发现纳入风险规避的方法通常能提高性能。最后，我们分析了LLM-as-a-judge与思维链（CoT）提示相结合的情况，表明CoT可能压缩判断分布的离散度，常常损害性能。我们的发现表明，利用分布输出比仅使用文本界面更能改进LLM-as-a-judge。
英文摘要：
Using language models to scalably approximate human preferences on text quality (LLM-as-a-judge) has become a standard practice applicable to many tasks. A judgment is often extracted from the judge's textual output alone, typically with greedy decoding. However, LLM judges naturally provide distributions over judgment tokens, inviting a breadth of inference methods for extracting fine-grained preferences. We find that taking the mean of the judgment distribution consistently outperforms taking the mode (i.e. greedy decoding) in all evaluation settings (i.e. pointwise, pairwise, and listwise). We further explore novel methods of deriving preferences from judgment distributions, and find that methods incorporating risk aversion often improve performance. Lastly, we analyze LLM-as-a-judge paired with chain-of-thought (CoT) prompting, showing that CoT can collapse the spread of the judgment distribution, often harming performance. Our findings show that leveraging distributional output improves LLM-as-a-judge, as opposed to using the text interface alone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
