In current benchmarks for evaluating large language models (LLMs), there are issues such as evaluation content restriction, untimely updates, and lack of optimization guidance. In this paper, we propose a new paradigm for the measurement of LLMs: Benchmarking-Evaluation-Assessment. Our paradigm shifts the "location" of LLM evaluation from the "examination room" to the "hospital". Through conducting a "physical examination" on LLMs, it utilizes specific task-solving as the evaluation content, performs deep attribution of existing problems within LLMs, and provides recommendation for optimization.
IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
The Conversational Artificial Intelligence (CoAI) Group, Tsinghua University(清华大学对话人工智能(CoAI)小组);Zhipu AI(智谱AI);University of Electronic Science and Technology of China(电子科技大学)
HarmMetric Eval: Benchmarking Metrics and Judges for LLM Harmfulness Assessment
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Zhejiang University(浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Pretraining Team, xAI(预训练团队,xAI);Hong Kong University of Science and Technology (GuangZhou)(香港科技大学(广州))
Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning
Department of Medical Engineering(医学工程系);Electrical Engineering Department(电气工程系);Sharif University of Technology(谢里夫技术大学);Department of Electrical Engineering(电气工程系);University of Tehran(德黑兰大学)
DS@GT at eRisk 2025: From prompts to predictions, benchmarking early depression detection with conversational agent based assessments and temporal attention models
SciHorizon: Benchmarking AI-for-Science Readiness from Scientific Data to Large Language Models
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学);Hefei University of Technology(合肥工业大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
PaperAudit-Bench: Benchmarking Error Detection in Research Papers for Critical Automated Peer Review
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Pengcheng Laboratory(鹏城实验室);University of Chinese Academy of Sciences(中国科学院大学);Wenge Technology(文英科技)
Vote'n'Rank: Revision of Benchmarking with Social Choice Theory
HSE University(高等经济大学);SaluteDevices(SaluteDevices公司);University of Oxford(牛津大学);Artificial Intelligence Research Institute(人工智能研究院);Institute of Linguistics, RAS(俄罗斯科学院语言学研究所);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心)