先做后评：自我参照作为提升大语言模型评估的途径
Do Before You Judge: Self-Reference as a Pathway to Better LLM Evaluation
AI总结：
本研究通过多模型多任务分析发现LLM生成与判断能力弱相关，提出利用模型自身答案作为参照的自我参照评估策略，显著增强两者相关性，为模型选择提供可靠代理。
摘要：
LLM-as-Judge框架在人工智能评估中日益流行，然而关于模型生成能力与判断能力之间关系的研究发现仍不一致。我们通过跨11个模型和21个多样化任务的系统性数据集级和实例级分析来研究这一关系。尽管这两种能力依赖于相同的基础知识，我们的分析揭示它们仅弱相关，主要原因是LLMs对被评判响应的敏感性。为解决此问题，我们提出一种自我参照引导的评估策略，利用模型自身的答案作为参照。该方法显著增强了生成能力与判断能力之间的相关性，提供了一条对齐这些技能的实用路径，并为评估任务中的模型选择提供了可靠代理。
英文摘要：
LLM-as-Judge frameworks are increasingly popular for AI evaluation, yet research findings on the relationship between models' generation and judgment abilities remain inconsistent. We investigate this relationship through systematic dataset- and instance-level analyses across 11 models and 21 diverse tasks. Despite both capabilities relying on the same underlying knowledge, our analyses reveal they are only weakly correlated, primarily due to LLMs' sensitivity to the responses being judged. To address this, we propose a self-reference-guided evaluation strategy that leverages a model's own answers as references. This approach significantly strengthens the correlation between generation and judgment abilities, offering a practical path to align these skills and providing a reliable proxy for model selection in evaluation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
