超越单点判断：面向LLM-as-a-Judge的分布对齐
Beyond Single-Point Judgment: Distribution Alignment for LLM-as-a-Judge
AI总结：
针对LLM-as-a-Judge范式下现有单点评估忽略人工评估多样性与不确定性的问题，提出结合KL散度对齐、交叉熵正则与对抗训练的分布对齐框架，提升评估质量、准确率与鲁棒性。
摘要：
在LLM-as-a-Judge（大语言模型作为评判者）范式中，LLM已成为强大的评估工具，与人工评判相比具备显著的效率与灵活性优势。但现有方法主要依赖单点评估，忽略了人工评估中固有的多样性与不确定性，会导致信息损失并降低评估可靠性。为解决该局限，我们提出一种新型训练框架，可显式将LLM生成的判断分布与经验人工分布对齐。具体而言，我们提出基于KL散度的分布对齐目标，结合辅助交叉熵正则项以稳定训练过程。此外，考虑到经验分布可能来自有限的人工标注，我们引入对抗训练以增强模型对分布扰动的鲁棒性。在多种LLM骨干模型与评估任务上的大量实验表明，我们的框架显著优于现有闭源LLM与传统单点对齐方法，在对齐质量、评估准确率与鲁棒性上均有提升。
英文摘要：
LLMs have emerged as powerful evaluators in the LLM-as-a-Judge paradigm, offering significant efficiency and flexibility compared to human judgments. However, previous methods primarily rely on single-point evaluations, overlooking the inherent diversity and uncertainty in human evaluations. This approach leads to information loss and decreases the reliability of evaluations. To address this limitation, we propose a novel training framework that explicitly aligns the LLM-generated judgment distribution with empirical human distributions. Specifically, we propose a distributional alignment objective based on KL divergence, combined with an auxiliary cross-entropy regularization to stabilize the training process. Furthermore, considering that empirical distributions may derive from limited human annotations, we incorporate adversarial training to enhance model robustness against distribution perturbations. Extensive experiments across various LLM backbones and evaluation tasks demonstrate that our framework significantly outperforms existing closed-source LLMs and conventional single-point alignment methods, with improved alignment quality, evaluation accuracy, and robustness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
