Reward models are central to preference alignment for legal tasks, yet existing reward model benchmarks are still primarily centered on general-domain preferences. In this work, we introduce LegalRewardBench, the first human-verified benchmark for evaluating reward models in the legal domain. The benchmark organizes legal preference evaluation into five capability levels: knowledge, understanding, analysis, decision, and writing, and preference examples are constructed through rubric-guided automatic judging combined with human verification. We evaluate three families of judges, covering both general and fine-tuned models, and generative and discriminative architectures. Experimental results show that strong general-purpose generative models are currently the most robust legal reward models, while existing fine-tuned reward models transfer less reliably to legal preference evaluation. We also conduct detailed comparisons of input organizations and training strategies.
AR-BENCH: Benchmarking Legal Reasoning with Judgment Error Detection, Classification and Correction
SKLCCSE Beihang University(北京航空航天大学软件学院);SCCE University of Science and Technology Beijing(北京科技大学计算机学院);Sino-French Engineer School Beihang University(北京航空航天大学中法工程师学院);People’s Procuratorate of Beijing Municipality(北京市人民检察院)
EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction
Division of Computer Science, School of Electrical and Computer Engineering(计算机科学系,电气与计算机工程学院);National Technical University of Athens(雅典技术大学);Department of Humanities Social Sciences and Law, School of Applied Mathematical and Physical Sciences(人文社会科学与法律系,应用数学与物理科学学院)
One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Support
University of Bern(伯尔尼大学);University of Copenhagen(哥本哈根大学);Bern University of Applied Sciences(伯尔尼应用科学大学);University of Fribourg(弗里堡大学);Stanford University(斯坦福大学)
LEXam: Benchmarking Legal Reasoning on 340 Law Exams
ETH Zurich(苏黎世联邦理工学院);University of Zurich(苏黎世大学);University of Lausanne(洛桑大学);Max Planck Institute for Research on Collective Goods(马克斯·普朗克集体物品研究所);Omnilex;University of St. Gallen(圣加仑大学);Swiss Federal Supreme Court(瑞士联邦最高法院)
LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases
Tsinghua University(清华大学);Peking University(北京大学);University of Glasgow(格拉斯哥大学);Beijing University of Posts and Telecommunications(北京邮电大学);Queen Mary University of London(伦敦玛丽女王大学);Northeastern University(东北大学)
LegalCiteTrust: Benchmarking Citation Trustworthiness in Chinese Long-Form Legal Research Reports
Faculty of Data Science, City University of Macau(澳门城市大学数据科学学院);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Southern University of Science and Technology(南方科技大学)
The Measurement Gap in the Automation of EU Law: Benchmarking Doctrinal Legal Reasoning under the EU AI Act
Chair of Law and Artificial Intelligence and Director, CZS Institute for Artificial Intelligence and Law, University of Tübingen(法律与人工智能教授、人工智能与法律研究所主任,图宾根大学)
ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation
University of Science and Technology of China(中国科学技术大学);Singapore Management University(新加坡管理大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
BenGER: Benchmarking LLM Systems on Subsumption-Based Legal Reasoning in German Law
Technical University of Munich (TUM)(慕尼黑技术大学);Ludwig Maximilian University of Munich (LMU)(慕尼黑路德维希-马克西米利安大学);University of Konstanz(康斯坦茨大学);University of Saarbrücken(萨尔布吕肯大学)
Validate Your Authority: Benchmarking LLMs on Multi-Label Precedent Treatment Classification
Department of Information Science and Technology(信息科学与技术系);College of Emergency Preparedness, Homeland Security, and Cybersecurity(应急准备、国土安全与网络安全学院);University at Albany, SUNY(萨利纳大学)
Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study
Department of Data Science & AI, Monash University(数据科学与人工智能系,莫纳什大学);Faculty of Law, Monash University(法学院,莫纳什大学)
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Google DeepMind(谷歌DeepMind);University of Alabama in Huntsville(阿拉巴马大学亨茨维尔分校);Google(谷歌);University of Missouri Columbia(密苏里大学哥伦比亚分校);Chunghwa Telecom Laboratories(春木电信实验室);University of Southern California(南加州大学);Polytechnique Montreal(蒙特利尔理工学院);Nutanix;ThinkEvolve Labs(ThinkEvolve实验室);UCL(伦敦大学学院);Infocomm Media Development Authority(信息通信媒体发展局);Monark Health(Monark健康);Seoul National University(首尔国立大学);Microsoft(微软);Centre for Responsible AI (CeRAI), Wadhwani School of Data Science and AI (WSAI), Indian Institute of Technology Madras(负责任人工智能中心(CeRAI)、瓦达威人工智能学校(WSAI)、印度理工学院马德拉斯分校);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Microsoft Research India(微软印度研究院);Stanford University(斯坦福大学);Argonne National Laboratory(阿贡国家实验室);University of Oxford(牛津大学);KAIST(韩国科学技术院);Yonsei University(延世大学);Amazon(亚马逊);UIUC(伊利诺伊大学香槟分校);Rochester Institute of Technology(罗切斯特理工学院);Xenoscube Inc.(Xenoscube公司);Korea AI Safety Institute (K-AISI)(韩国人工智能安全研究所(K-AISI));MLCommons;CommonGround;Artifex Labs(Artifex实验室)
TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains
Shanghai Key Laboratory of Computer Software Testing and Evaluating, Shanghai Development Center of Computer Software Technology(上海软件测试与评估关键实验室、上海计算机软件技术发展中心);Shanghai Normal University(上海师范大学);Shanghai Artificial Intelligence Lab(上海人工智能实验室);Shanghai University(上海大学);Shanghai Jiao Tong University(上海交通大学)
IPBench: Benchmarking the Knowledge of Large Language Models in Intellectual Property
Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China(深圳高性能数据挖掘重点实验室,深圳先进技术研究院,中国科学院,中国);University of Chinese Academy of Sciences, China(中国科学院大学,中国);Dalian University of Technology, China(大连理工大学,中国);School of Biomedical Engineering, UNSW Sydney, Australia(生物医学工程学院,新南威尔士大学悉尼分校,澳大利亚);Shenzhen MSU-BIT University, China(深圳MSU-BIT大学,中国);Shenzhen University of Advanced Technology, China(深圳大学先进技术学院,中国)