Large Language Models (LLMs) are advancing rapidly, yet the benchmarks used to measure this progress are becoming increasingly unreliable. Score inflation and selective reporting have eroded the authority of standard benchmarks, leaving the community uncertain about which evaluation results remain trustworthy. We introduce the Benchmark Health Index (BHI), a pure data-driven framework for auditing evaluation sets along three orthogonal and complementary axes: (1) Capability Discrimination, measuring how sharply a benchmark separates model performance beyond noise; (2) Anti-Saturation, estimating remaining headroom before ceiling effects erode resolution and thus the benchmark's expected longevity; and (3) Impact, quantifying influence across academic and industrial ecosystems via adoption breadth and practice-shaping power. By distilling 106 validated benchmarks from the technical reports of 91 representative models in 2025, we systematically characterize the evaluation landscape. BHI is the first framework to quantify benchmark health at a macro level, providing a principled basis for benchmark selection and enabling dynamic lifecycle management for next-generation evaluation protocols.
HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring
School of Computing and Information Systems, University of Melbourne, Australia(墨尔本大学计算机与信息系统学院);School of Computer Science, University of Auckland, New Zealand(奥克兰大学计算机科学学院)
FAM-Bench: Benchmarking Condition-Aware Food-as-Medicine Reasoning in Vision-Language Models
Department of Electrical Engineering, University of South Florida(佛罗里达州立大学电气工程系);Muma College of Business, University of South Florida(佛罗里达州立大学Muma商学院);Computer Science, University of Copenhagen(哥本哈根大学计算机科学系)
PHM-Bench: A Domain-Specific Benchmarking Framework for Systematic Evaluation of Large Models in Prognostics and Health Management
Hangzhou International Innovation Institute, Beihang University(北京航空航天大学杭州国际创新学院);Institute of Reliability Engineering, Beihang University(北京航空航天大学可靠性工程研究所);Science & Technology on Reliability & Environmental Engineering Laboratory(可靠性与环境工程技术重点实验室);School of Reliability and Systems Engineering, Beihang University(北京航空航天大学可靠性与系统工程学院)
VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare
School of Computing and Information System, the University of Melbourne, Australia(墨尔本大学计算与信息学院);School of Computing, FSE, Macquarie University, Australia(麦考瑞大学计算学院)
Benchmarking Foundation Models with Multimodal Public Electronic Health Records
Centre for Quantitative Medicine and Duke-NUS AI + Medical Science Initiative, Duke-NUS Medical School(定量医学中心和杜克-国立新加坡大学AI+医学科学计划,杜克-国立新加坡大学医学院);Graduate School of Engineering, The University of Tokyo(东京大学工程研究生院);Department of Population Health Sciences, Weill Cornell Medicine(流行病学与公共卫生科学系,韦尔·科恩医学中心);Department of Surgery, Duke University School of Medicine(外科医学系,杜克大学医学学院);Centre for Quantitative Medicine, Duke-NUS AI + Medical Science Initiative and Programme in Health Services and Systems Research, Duke-NUS Medical School and NUS Artificial Intelligence Institute, National University of Singapore(定量医学中心和杜克-国立新加坡大学AI+医学科学计划及健康服务与系统研究计划,杜克-国立新加坡大学医学院和新加坡国立大学人工智能研究所)
Generating Synthetic Electronic Health Record Data: a Methodological Scoping Review with Benchmarking on Phenotype Data and Open-Source Software
Department of Biostatistics, University of Michigan(生物统计学系,密歇根大学);Department of Biomedical Informatics and Data Science, Yale University(生物医学信息学与数据科学系,耶鲁大学);Department of Biostatistics, Yale University(生物统计学系,耶鲁大学)
SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats
University of Chinese Academy of Sciences(中国科学院大学);Tsinghua University(清华大学);Beijing University of Posts and Telecommunications(北京邮电大学);University of Science and Technology of China(中国科学技术大学)
Power Transformer Health Index and Life Span Assessment: A Comprehensive Review of Conventional and Machine Learning based Approaches
HITEC University(HITEC大学);Riphah International University(里法国际大学);Pak-Austria Fachhochschule: Institute of Applied Sciences and Technology(巴基斯坦-奥地利应用科学与技术学院);Robert Gordon University(罗伯特戈登大学)
LingxiDiagBench: A Multi-Agent Framework for Benchmarking LLMs in Chinese Psychiatric Consultation and Diagnosis
Tianqiao and Chrissy Chen Institute(天桥和克里斯西·陈研究所);EverMind AI Inc.(EverMind AI公司);Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine(上海精神卫生中心,上海交通大学医学院)