UDA：用于成对LLM-as-a-Judge的无监督去偏对齐
UDA: Unsupervised Debiasing Alignment for Pair-wise LLM-as-a-Judge
AI总结：
针对成对LLM-as-a-Judge评估中的偏好偏差问题，提出无监督去偏对齐框架UDA，通过动态调整Elo评分系统减少评判模型分歧，实验显著降低评分标准差并提升与人类判断的相关性，优化评估生态系统。
摘要：
大语言模型（LLMs）的成对评估是一种常见范式，但易受偏好偏差影响，即评判模型会系统性偏好某些输出（如自身输出）。这种偏差导致不同评判模型间的排名不一致且有偏差。为解决该问题，我们首先通过实证证明跨模型评估中存在显著且异质的偏差。然后提出UDA（无监督去偏对齐）框架，通过动态调整Elo评分系统减少评判模型间的分歧。对于每对比较，一个紧凑的神经网络学习自适应设置K因子并优化获胜概率。关键是，UDA以完全无监督方式运行，仅以最小化所有评判模型Elo轨迹的离散度为目标。这促使模型向集体共识对齐，该共识作为更稳定且可复现评估的无监督代理。此外，我们提供理论动机，证明向共识对齐如何减少聚合系统偏差。实验表明，UDA将评判模型间评分标准差显著降低多达63.4%，并将与人类判断的平均相关性提高24.7%。值得注意的是，UDA提升了表现不佳的评判模型的性能，使其与高质量模型持平，促进了更稳健可靠的评估生态系统。代码和数据可在https://anonymous.4open.science/r/62AB93CD-23B4获取。
英文摘要：
Pairwise evaluation of Large Language Models (LLMs) is a common paradigm, but it is prone to preference bias, where judges systematically favor certain outputs, such as their own. This bias leads to inconsistent and skewed rankings across different judges. To address this, we first empirically demonstrate significant and heterogeneous biases in cross-model evaluations. We then propose UDA (Unsupervised Debiasing Alignment), a framework that reduces inter-judge disagreement by dynamically adjusting the Elo rating system. For each pairwise comparison, a compact neural network learns to adaptively set the K-factor and refine win probabilities. Crucially, UDA operates in a fully unsupervised manner, guided solely by the objective of minimizing the dispersion among the Elo trajectories of all judges. This forces an alignment towards a collective consensus, which serves as an unsupervised proxy for a more stable and reproducible evaluation. In addition, we provide theoretical motivation demonstrating how alignment towards a consensus can reduce aggregate system bias. Experiments show that UDA significantly reduces the inter-judge rating standard deviation by up to 63.4% and improves the average correlation with human judgments by 24.7%. Notably, UDA elevates the performance of poorly performing judges to achieve parity with high-quality ones, fostering a more robust and reliable evaluation ecosystem. Code and data are available at https://anonymous.4open.science/r/62AB93CD-23B4.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
