群体比较推理：解锁LLM-as-a-Judge的全面评估
Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge
AI总结：
本文提出基于群体比较的评估方法，通过引入额外群体响应与候选响应比较，引导LLM-as-a-Judge生成更全面的思维链判断，在五个基准上平均准确率提升6.7%，并支持更高效的监督微调。
摘要：
LLM-as-a-Judge通过生成思维链（CoT）判断，已成为广泛采用的自动评估方法。然而，其可靠性因CoT推理无法捕捉全面且更深层的细节而受损，往往导致不完整的结果。现有方法主要依赖多数投票或标准扩展，不足以解决CoT中的这一局限。我们提出基于群体的比较评估方法，引入额外的群体响应与候选响应进行比较，从而揭示候选响应中更深层、更全面的细节。该过程有效引导LLM-as-a-Judge提供更详细的CoT判断。大量实验表明，我们的方法提升了评估可靠性，在五个基准上平均准确率提高了6.7%。此外，我们的方法生成了更高质量的CoT，有助于判断器蒸馏，并在用于监督微调（SFT）的拒绝采样中表现出优越性能，称为群体拒绝采样，从而实现更高效的SFT。我们的分析证实，我们生成的CoT更全面且质量更高，且评估准确率随着推理规模的扩大而提高。
英文摘要：
LLM-as-a-Judge, which generates chain-of-thought (CoT) judgments, has become a widely adopted auto-evaluation method. However, its reliability is compromised by the CoT reasoning's inability to capture comprehensive and deeper details, often leading to incomplete outcomes. Existing methods mainly rely on majority voting or criteria expansion, which is insufficient to address the limitation in CoT. We propose Crowd-based Comparative Evaluation, which introduces additional crowd responses to compare with the candidate responses, thereby exposing deeper and more comprehensive details within the candidate responses. This process effectively guides LLM-as-a-Judge to provide a more detailed CoT judgment. Extensive experiments demonstrate that our approach enhances evaluation reliability, achieving an average accuracy gain of 6.7% across five benchmarks. Moreover, our method produces higher-quality CoTs that facilitate judge distillation and exhibit superior performance in rejection sampling for supervised fine-tuning (SFT), referred to as crowd rejection sampling, thereby enabling more efficient SFT. Our analysis confirms that CoTs generated by ours are more comprehensive and of higher quality, and evaluation accuracy improves as inference scales.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
