JudgeLM：微调大型语言模型是可扩展的评判器
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
AI总结：
提出JudgeLM，通过微调LLMs构建可扩展评判器，利用大规模数据集和偏差缓解技术，在多基准上实现超越人类一致率的最先进评判性能。
摘要：
在开放式场景中评估大型语言模型（LLMs）具有挑战性，因为现有的基准和指标无法全面衡量它们。为了解决这个问题，我们提出微调LLMs作为可扩展的评判器（JudgeLM），以在开放式基准中高效且有效地评估LLMs。我们首先提出了一个全面、大规模、高质量的数据集，包含任务种子、LLMs生成的答案和GPT-4生成的评判，用于微调高性能评判器，以及一个用于评估评判器的新基准。我们在不同规模（从7B、13B到33B参数）上训练JudgeLM，并对其能力和行为进行了系统分析。然后，我们分析了将LLM微调为评判器时的关键偏差，将其考虑为位置偏差、知识偏差和格式偏差。为了解决这些问题，JudgeLM引入了一系列技术，包括交换增强、参考支持和参考丢弃，这些技术显著提升了评判器的性能。JudgeLM在现有的PandaLM基准和我们提出的新基准上均获得了最先进的评判器性能。我们的JudgeLM高效，JudgeLM-7B仅需3分钟即可用8块A100 GPU评判5K个样本。JudgeLM与教师评判器高度一致，达成的一致率超过90%，甚至超越了人与人之间的一致率。JudgeLM还展示了在单答案、多模态模型、多答案、多轮对话等评判中的扩展能力。代码可在https://github.com/baaivision/JudgeLM获取。
英文摘要：
Evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively. To address this problem, we propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks. We first propose a comprehensive, large-scale, high-quality dataset containing task seeds, LLMs-generated answers, and GPT-4-generated judgments for fine-tuning high-performance judges, as well as a new benchmark for evaluating the judges. We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors. We then analyze the key biases in fine-tuning LLM as a judge and consider them as position bias, knowledge bias, and format bias. To address these issues, JudgeLM introduces a bag of techniques including swap augmentation, reference support, and reference drop, which clearly enhance the judge's performance. JudgeLM obtains the state-of-the-art judge performance on both the existing PandaLM benchmark and our proposed new benchmark. Our JudgeLM is efficient and the JudgeLM-7B only needs 3 minutes to judge 5K samples with 8 A100 GPUs. JudgeLM obtains high agreement with the teacher judge, achieving an agreement exceeding 90% that even surpasses human-to-human agreement. JudgeLM also demonstrates extended capabilities in being judges of the single answer, multimodal models, multiple answers, multi-turn chat, etc. Code is available at https://github.com/baaivision/JudgeLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
