arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.10669cs.CL
收藏

人类还是LLM作为裁判?关于评判偏差的研究

Humans or LLMs as the Judge? A Study on Judgement Biases

The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院)

Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, Benyou Wang

AI总结:

本文提出一个无需真实标注的框架,通过定制数据集调查人类与LLM裁判在四种偏差上的表现,发现两者均存在显著脆弱性,并利用这些偏差成功对LLM裁判发起攻击,呼吁开发更鲁棒的评估系统。

摘要:

采用人类和大型语言模型(LLM)作为裁判(即 human-as-a-judge 和 LLM-as-a-judge)来评估 LLM 的性能近期备受关注。然而,该方法同时引入了来自人类和 LLM 的潜在偏差,质疑了评估结果的可靠性。本文提出一个无需参考真实标注的新框架,用于调查 LLM 和人类裁判上的 Misinformation Oversight Bias、Gender Bias、Authority Bias 和 Beauty Bias。我们策划了一个参考修订版 Bloom's Taxonomy 的数据集,并进行了数千次评估。结果显示,人类和 LLM 裁判在不同程度上易受干扰,即使是最前沿的裁判也存在相当大的偏差。我们进一步利用这些偏差对 LLM 裁判发起攻击。希望本研究能提醒社区关注这些裁判的偏差与脆弱性,以及开发鲁棒评估系统的紧迫性。

英文摘要:

Adopting human and large language models (LLM) as judges (a.k.a human- and LLM-as-a-judge) for evaluating the performance of LLMs has recently gained attention. Nonetheless, this approach concurrently introduces potential biases from human and LLMs, questioning the reliability of the evaluation results. In this paper, we propose a novel framework that is free from referencing groundtruth annotations for investigating Misinformation Oversight Bias, Gender Bias, Authority Bias and Beauty Bias on LLM and human judges. We curate a dataset referring to the revised Bloom's Taxonomy and conduct thousands of evaluations. Results show that human and LLM judges are vulnerable to perturbations to various degrees, and that even the cutting-edge judges possess considerable biases. We further exploit these biases to conduct attacks on LLM judges. We hope that our work can notify the community of the bias and vulnerability of human- and LLM-as-a-judge, as well as the urgency of developing robust evaluation systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑