arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02898cs.AIcs.CL
收藏

将语言模型基准与成对偏好对齐

Aligning Language Model Benchmarks with Pairwise Preferences

School of Data Science, University of Virginia(弗吉尼亚大学数据科学学院);Imperial College London(伦敦帝国理工学院);Thomson Reuters Foundational Research(汤姆森路透基础研究);Department of Electrical Engineering and Computer Science, UC Berkeley and UCSF(伯克利大学电气工程与计算机科学系及旧金山大学)

Marco Gutierrez, Xinyi Leng, Hannah Cyberey, Jonathan Richard Schwarz, Ahmed Alaa, Thomas Hartvigsen

AI总结:

提出BenchAlign方法,通过利用语言模型在问题级别的性能与模型成对排名,自动调整离线基准权重,使新基准能根据偏好准确排序未见模型。

摘要:

语言模型基准是广泛使用的、计算高效的现实性能代理。然而,许多近期工作发现基准常常无法预测实际效用。为弥合这一差距,我们引入基准对齐,即利用有限的模型性能信息自动更新离线基准,旨在生成新的静态基准,以预测给定测试设置中的模型成对偏好。然后我们提出BenchAlign,这是该问题的首个解决方案,它利用语言模型在问题级别的性能以及可能在部署期间收集的模型成对排名,学习基准问题的偏好对齐权重,生成新的基准,根据这些偏好对先前未见过的模型进行排序。我们的实验表明,我们的对齐基准能够根据人类偏好模型准确地对未见模型进行排序,即使模型大小不同,同时保持可解释性。总体而言,我们的工作为将基准与实际人类偏好对齐的局限性提供了见解,这有助于加速模型开发以追求实际效用。

英文摘要:

Language model benchmarks are pervasive and computationally-efficient proxies for real-world downstream performance. However, many recent works find that benchmarks often fail to predict downstream utility. While some works have begun diagnosing sources of misalignment, there remain no ways to systematically update benchmarks to align their scores with downstream usage. Towards bridging this gap, we introduce and study \textit{benchmark alignment}, where we use information about downstream model performance to automatically update benchmarks, specifically aiming to update static benchmarks so they generalizably rank models according to new pairwise preferences. Our experiments involving 4576 language models and 6 benchmarks show that reweighting benchmark items can successfully rank unseen models, even generalizing across model scales in most cases. And while naive alignment unsurprisingly requires large numbers of models and benchmark questions, an oracle experiment suggests this could be reduced to as few as 20 well-chosen models. Overall, our work takes a step towards efficiently aligning benchmark development with downstream tasks.\footnote{All of our code, models, and data are publicly-available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑