对齐模型评估与人类偏好：缓解语言模型评估中的令牌计数偏差
Aligning Model Evaluations with Human Preferences: Mitigating Token Count Bias in Language Model Assessments
AI总结：
本研究针对语言模型评估中令牌计数偏差，提出贝叶斯统计和 t 检验量化及重新校准 GPTScorer 的方法，显著提升与人类评估的一致性，如推荐用例相关系数从 -27.27 升至 44.55。
摘要：
SLAM 论文表明，设备端小型语言模型（SLMs）是 API 基础大型语言模型（LLMs）（如 OpenAI 的 GPT-4）的可行且经济高效的替代方案，提供相当的性能和稳定性。然而，SLAM 也发现了人类偏好与传统自动评估器之间的差异。这篇后续论文探讨了通过解决偏差（特别是对更高令牌计数的偏好）来使 LLM 评估器与人类评估对齐的方法。我们采用贝叶斯统计和 t 检验来量化这种偏差，并开发了一种重新校准程序来调整 GPTScorer。我们的发现显著改善了重新校准的 LLM 评估器在多个用例中与人类评估的对齐。例如，在推荐用例中，斯皮尔曼排名相关系数从 -27.27 提高到 44.55。这些结果强调了在自动评估中考虑偏差以确保公平和准确的模型评估的重要性。重新校准过程增强了自动评估器的可靠性，从而产生与人类价值观和期望一致的更好的 AI 模型。这项研究为未来偏差纠正研究提供了稳健的方法论，并强调了开发人类对齐的 AI 评估系统的可行性和益处。
英文摘要：
The SLAM paper demonstrated that on-device Small Language Models (SLMs) are a viable and cost-effective alternative to API-based Large Language Models (LLMs), such as OpenAI's GPT-4, offering comparable performance and stability. However, SLAM also identified discrepancies between human preferences and traditional auto-evaluators. This follow-up paper explores methods to align LLM evaluator preferences with human evaluations by addressing biases, particularly toward higher token counts. We employed Bayesian statistics and a t-test to quantify this bias and developed a recalibration procedure to adjust the GPTScorer. Our findings significantly improve aligning the recalibrated LLM evaluator with human evaluations across multiple use cases. For instance, spearman's ranking correlation score in the Recommendation use case improved from -27.27 to 44.55. These results highlight the importance of accounting for biases in automated evaluations to ensure fair and accurate model assessments. The recalibration process enhances the reliability of automated evaluators, leading to better AI models that align with human values and expectations. This study provides a robust methodology for future research into bias correction and emphasizes the feasibility and benefits of developing human-aligned AI evaluation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
