评估法官作为评估者：JETTS基准——LLM作为法官在测试时扩展评估中的表现
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
AI总结：
本文提出JETTS基准，评估LLM法官在测试时扩展中的表现，发现其在重排序中与结果奖励模型相当，但在束搜索中不如过程奖励模型，且自然语言批评对改进生成效果有限。
摘要：
扩展测试时计算，即在推理过程中为生成式大语言模型（LLM）提供额外的计算资源，通常依赖于外部非生成式评估器（即奖励模型）的帮助。与此同时，LLM法官——即经过训练以自然语言生成评估和批评（解释）的模型——在自动评估中日益流行。尽管法官在经验上取得了成功，但它们在测试时扩展设置中作为评估者的有效性在很大程度上仍是未知的。在本文中，我们引入了测试时扩展的法官评估（JETTS）基准，该基准在三个领域（数学推理、代码生成和指令遵循）中评估法官性能，并涵盖三种任务设置：响应重排序、步骤级束搜索和基于批评的响应改进。我们评估了10个不同的法官模型（7B-70B参数）和8个不同的基础生成模型（6.7B-72B参数）。我们的基准表明，虽然法官在重排序中与结果奖励模型具有竞争力，但在束搜索过程中它们始终不如过程奖励模型。此外，尽管法官具有独特性，但其自然语言批评目前在引导生成器产生更好响应方面效果不佳。
英文摘要：
Scaling test-time computation, or affording a generator large language model (LLM) extra compute during inference, typically employs the help of external non-generative evaluators (i.e., reward models). Concurrently, LLM-judges, models trained to generate evaluations and critiques (explanations) in natural language, are becoming increasingly popular in automatic evaluation. Despite judge empirical successes, their effectiveness as evaluators in test-time scaling settings is largely unknown. In this paper, we introduce the Judge Evaluation for Test-Time Scaling (JETTS) benchmark, which evaluates judge performance in three domains (math reasoning, code generation, and instruction following) under three task settings: response reranking, step-level beam search, and critique-based response refinement. We evaluate 10 different judge models (7B-70B parameters) for 8 different base generator models (6.7B-72B parameters). Our benchmark shows that while judges are competitive with outcome reward models in reranking, they are consistently worse than process reward models in beam search procedures. Furthermore, though unique to LLM-judges, their natural language critiques are currently ineffective in guiding the generator towards better responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
