arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.04313cs.LGcs.AIcs.CL
收藏

优秀模型的思维趋同及其对AI监督的削弱作用

Great Models Think Alike and this Undermines AI Oversight

Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);Stanford University Core contributors(斯坦福大学核心贡献者)

Shashwat Goel, Joschka Struber, Ilze Amanda Auzina, Karuna K Chandra, Ponnurangam Kumaraguru, Douwe Kiela, Ameya Prabhu, Matthias Bethge, Jonas Geiping

AI总结:

本研究提出基于模型错误重合度的CAPA指标衡量语言模型相似性,发现LLM-as-a-judge评分偏向相似模型、弱强模型互补知识影响弱到强泛化,且能力越强的模型错误越相似,凸显AI监督中校正模型相似性的重要性。

摘要:

随着语言模型(LM)能力的提升,人类对其进行大规模评估与监督的难度日益增加。有观点期望其他语言模型能够自动化完成这两项任务,我们将该模式称为“AI Oversight(AI监督)”。我们提出了Chance Adjusted Probabilistic Agreement(CAPA,机会调整概率一致性)——一种基于模型错误重合度衡量LM相似性的指标,以此研究模型相似性对AI监督两大方面的影响。借助CAPA,我们首先证明LLM-as-a-judge(大语言模型作为评判者)评分会偏向与评判者相似的模型,这一发现推广了近期的自偏好相关结果。随后我们研究了基于LM标注的训练,发现弱监督者与强学生模型之间的互补知识是“弱到强泛化”收益的关键影响因素。随着模型能力提升,发现其错误的难度增加,我们可能会更多依赖AI监督。但我们观察到一个令人担忧的趋势:模型能力越强,其错误模式越相似,这指向了关联失效带来的风险。本研究强调了在AI监督这一新兴范式中,报告并校正模型相似性的重要性。

英文摘要:

As Language Model (LM) capabilities advance, evaluating and supervising them at scale is getting harder for humans. There is hope that other language models can automate both these tasks, which we refer to as ''AI Oversight''. We study how model similarity affects both aspects of AI oversight by proposing Chance Adjusted Probabilistic Agreement (CAPA): a metric for LM similarity based on overlap in model mistakes. Using CAPA, we first show that LLM-as-a-judge scores favor models similar to the judge, generalizing recent self-preference results. Then, we study training on LM annotations, and find complementary knowledge between the weak supervisor and strong student model plays a crucial role in gains from ''weak-to-strong generalization''. As model capabilities increase, it becomes harder to find their mistakes, and we might defer more to AI oversight. However, we observe a concerning trend -- model mistakes are becoming more similar with increasing capabilities, pointing to risks from correlated failures. Our work underscores the importance of reporting and correcting for model similarity, especially in the emerging paradigm of AI oversight.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑