arXiv 2509.110262025-09-14cs.AIcs.CL

重新思考LLM推理依据的人类偏好评估

Rethinking Human Preference Evaluation of LLM Rationales

University of Washington（华盛顿大学）；Stanford University（斯坦福大学）

Ziang Li, Manasi Ganti, Zixian Ma, Helena Vasconcelos, Qijia He, Ranjay Krishna

备注：: Published in the XLLM-Reason-Plan Workshop on the Application of LLM Explainability to Reasoning and Planning at COLM 2025
发表信息：: Proceedings of the XLLM-Reason-Plan Workshop, Conference on Language Modeling (COLM) 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出基于属性的细粒度评估方法，通过SHAP分析MT Bench和Chatbot Arena数据集，揭示了LLM推理依据质量的关键属性，并使用特定属性的ELO分数提供更细致的模型比较。

摘要：

大型语言模型（LLMs）通常生成自然语言推理依据——即自由形式的解释，有助于提升复杂推理任务上的表现并增强对人类用户的可解释性。然而，评估这些推理依据仍具挑战性。尽管近期研究依赖来自人类或LLM评判者的二元偏好判断，此类评估往往不透明且粒度粗糙，对何种因素使一种推理依据优于另一种提供的洞察有限。在本工作中，我们通过以下问题重新思考对LLM生成推理依据的偏好评估：（1）哪些属性定义了好的推理依据？（2）人类偏好能否由这些属性解释？（3）基于属性的评估能否克服二元比较的局限？我们从先前文献中识别出一组关键推理依据属性，并使用自动指标、LLM判断和人类标注对其进行评估。随后，我们利用SHAP分析两个人类偏好数据集MT Bench和Chatbot Arena，以识别最能解释人类偏好结果的属性。最后，我们使用特定属性的ELO分数重新评估模型生成的推理依据，揭示了更细致的模型比较与洞察。研究结果表明，细粒度的属性评估能更好地表征推理依据质量，并引导未来研究走向更具可解释性和可靠性的评估实践。

英文摘要：

Large language models (LLMs) often generate natural language rationales -- free-form explanations that help improve performance on complex reasoning tasks and enhance interpretability for human users. However, evaluating these rationales remains challenging. While recent work has relied on binary preference judgments from humans or LLM judges, such evaluations are often opaque and coarse-grained, offering limited insight into what makes one rationale better than another. In this work, we rethink preference evaluation for LLM-generated rationales by asking: (1) What attributes define good rationales? (2) Can human preferences be explained by these attributes? (3) Can attribute-based evaluation overcome the limitations of binary comparisons? We identify a set of key rationale attributes from prior literature and assess them using automatic metrics, LLM judgments, and human annotations. We then analyze two standard human preference datasets MT Bench and Chatbot Arena using SHAP to identify which attributes best explain human preference outcomes. Finally, we re-evaluate model-generated rationales using attribute-specific ELO scores, revealing more nuanced model comparisons and insights. Our findings suggest that fine-grained attribute evaluations can better characterize rationale quality and guide future research toward more interpretable and reliable evaluation practices.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。