Large Language Models (LLMs) are powerful zero-shot assessors used in real-world situations such as assessing written exams and benchmarking systems. Despite these critical applications, no existing work has analyzed the vulnerability of judge-LLMs to adversarial manipulation. This work presents the first study on the adversarial robustness of assessment LLMs, where we demonstrate that short universal adversarial phrases can be concatenated to deceive judge LLMs to predict inflated scores. Since adversaries may not know or have access to the judge-LLMs, we propose a simple surrogate attack where a surrogate model is first attacked, and the learned attack phrase then transferred to unknown judge-LLMs. We propose a practical algorithm to determine the short universal attack phrases and demonstrate that when transferred to unseen models, scores can be drastically inflated such that irrespective of the assessed text, maximum scores are predicted. It is found that judge-LLMs are significantly more susceptible to these adversarial attacks when used for absolute scoring, as opposed to comparative assessment. Our findings raise concerns on the reliability of LLM-as-a-judge methods, and emphasize the importance of addressing vulnerabilities in LLM assessment methods before deployment in high-stakes real-world scenarios.
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation
Seoul National University(首尔大学);IPAI, Seoul National University(首尔大学IPAI);LG AI Research(LG人工智能研究院);NAVER AI Lab(NAVER人工智能实验室);NAVER Cloud(NAVER云);University of Richmond(里士满大学)
Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges
ZJU–UIUC Institute(浙大UIUC研究院);ZJU–Hangzhou Global Scientific and Technological Innovation Center(浙大杭州全球科技创新中心);College of Computer Science and Technology(计算机科学与技术学院);Alibaba Group(阿里巴巴集团)
Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judge
National Institute of Informatics (NII)(国立信息学研究所);University of Amsterdam(阿姆斯特丹大学);University of Tokyo(东京大学);Hitotsubashi University(一桥大学);Centrum Wiskunde & Informatica (CWI)(荷兰数学与计算机科学研究中心);Utrecht University(乌得勒支大学)
HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Zhejiang University(浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Pretraining Team, xAI(预训练团队,xAI);Hong Kong University of Science and Technology (GuangZhou)(香港科技大学(广州))
Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Baidu Inc.(百度公司);Beijing Key Laboratory of Big Data Management and Analysis Methods(北京市大数据管理与分析方法重点实验室)
AutoBench: Automating LLM Evaluation through Reciprocal Peer Assessment
Department of Computer Science, Sapienza University of Rome(计算机科学系,罗马萨皮恩扎大学);Department of Computer, Control, and Management Engineering, Sapienza University of Rome(计算机、控制与管理工程系,罗马萨皮恩扎大学);eZecute S.R.L.(eZecute公司)
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
Arizona State University(亚利桑那州立大学);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Northwestern University(西北大学);University of California, Berkeley(加州大学伯克利分校);Emory University(埃默里大学)
Using LLM-as-a-Judge/Jury to Advance Scalable, Clinically-Validated Safety Evaluations of Model Responses to Users Demonstrating Psychosis
Apart Research;Odyssean Institute;London School of Economics and Political Science(伦敦政治经济学院);Stanford Institute for Human-Centered AI(斯坦福大学以人为本人工智能研究所)
IDEA Research, International Digital Economy Academy(IDEA研究机构,国际数字经济学院);Sun Yat-sen University(孙中山大学);DataArc Tech Ltd(DataArc技术有限公司);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Department of Civil and Environmental Engineering, Imperial College London(伦敦帝国理工学院土木与环境工程系);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学硅谷人工智能学院);The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Department of Computer Science and Technology, Peking University(北京大学计算机科学与技术系)
JudgeAgent: Beyond Static Benchmarks for Knowledge-Driven and Dynamic LLM Evaluation
DataArc Tech Ltd.(DataArc科技有限公司);IDEA Research, International Digital Economy Academy(国际数字经济学院IDEA研究所);School of Advanced Interdisciplinary Sciences, UCAS(北京大学交叉学科学院);State Key Lab of AI Safety, Institute of Computing Technology, CAS(中国科学院计算技术研究所人工智能安全国家重点实验室)
University of California, Los Angeles(加州大学洛杉矶分校);Seoul National University(首尔国立大学);Qatar Computing Research Institute(卡塔尔计算研究所);Google(谷歌);George Washington University(乔治华盛顿大学);Stanford University(斯坦福大学);Scale AI;University of Washington(华盛顿大学)