与人类判断对齐：成对偏好在大语言模型评估器中的作用
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators
AI总结：
针对大语言模型评估器存在偏差、与人类评估对齐度不足的问题，本研究提出基于不确定性引导的成对偏好搜索排序聚合方法，在长文本生成评估任务上取得最优性能，并揭示了成对偏好在量化模型传递性中的作用。
摘要：
大语言模型（LLMs）作为自动评估器在评估生成自然语言的质量方面已展现出可观的能力。然而，LLMs在评估中仍存在偏差，且往往难以生成与人类评估一致的连贯评估结果。在本研究中，我们首先对LLM评估器与人类评估之间的不一致性开展了系统性研究，发现现有的旨在缓解LLMs偏差的校准方法不足以有效实现LLM评估器的对齐。受RLHF中偏好数据使用方式的启发，我们将评估问题构建为排序问题，并提出了成对偏好搜索（Pairwise-preference Search, PAIRS）——这是一种基于不确定性引导搜索的排序聚合方法，它利用LLMs在局部进行成对比较，并能高效地对候选文本进行全局排序。PAIRS在长文本生成的代表性评估任务上取得了当前最优（state-of-the-art）性能，且相比直接评分有显著提升。此外，我们还深入分析了成对偏好在量化LLMs传递性中的作用，并展示了PAIRS如何通过使用去偏差的成对评估进行校准而获益。
英文摘要：
Large Language Models (LLMs) have demonstrated promising capabilities as automatic evaluators in assessing the quality of generated natural language. However, LLMs still exhibit biases in evaluation and often struggle to generate coherent evaluations that align with human assessments. In this work, we first conduct a systematic study of the misalignment between LLM evaluators and human evaluation, revealing that existing calibration methods aimed at mitigating biases of LLMs are insufficient for effectively aligning LLM evaluators. Inspired by the use of preference data in RLHF, we formulate the evaluation as a ranking problem and introduce Pairwise-preference Search (PAIRS), an uncertainty-guided search-based rank aggregation method that employs LLMs to conduct pairwise comparisons locally and efficiently ranks candidate texts globally. PAIRS achieves state-of-the-art performance on representative evaluation tasks in long-form generations and demonstrates significant improvements over direct scoring. Furthermore, we provide insights into the role of pairwise preference in quantifying the transitivity of LLMs and demonstrate how PAIRS benefits from calibration using debiased pairwise evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
