Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research and have been developed for a variety of tasks ranging from question answering to facial recognition. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. In this paper, drawing upon research in moral philosophy and metaethics, we argue that it is impossible to develop such a benchmark. As such, alternative mechanisms are necessary for evaluating whether an AI system is 'ethical'. This is especially pressing in light of the prevalence of applied, industrial AI research. We argue that it makes more sense to talk about 'values' (and 'value alignment') rather than 'ethics' when considering the possible actions of present and future AI systems. We further highlight that, because values are unambiguously relative, focusing on values forces us to consider explicitly what the values are and whose values they are. Shifting the emphasis from ethics to values therefore gives rise to several new ways of understanding how researchers might advance research programmes for robustly safe or beneficial AI. We conclude by highlighting a number of possible ways forward for the field as a whole, and we advocate for different approaches towards more value-aligned AI research.
Crisis-Bench: Benchmarking Strategic Ambiguity and Reputation Management in Large Language Models
Hong Kong University of Science and Technology（香港科技大学）；Hong Kong Baptist University（香港 Baptist 大学）；National University of Singapore（新加坡国立大学）；Imperial College London（伦敦帝国理工学院）
Xi’an Jiaotong University（西安交通大学）；Singapore Management University（新加坡管理大学）；University of Southampton（南安普顿大学）；Harbin Institute of Technology（哈尔滨工业大学）；Pengcheng Laboratory（鹏城实验室）
PreAct-Bench: Benchmarking Predictive Monitoring in LLMs
King’s College London（伦敦国王学院）；National University of Singapore（新加坡国立大学）；Southern University of Science and Technology（南方科技大学）；Thomson Reuters Foundational Research（汤姆森路透基础研究）；Imperial College London（伦敦帝国学院）；The Alan Turing Institute（艾伦·图灵研究所）
Beyond One World: Benchmarking Super Heros in Role-Playing Across Multiversal Contexts
Faculty of Engineering, Thammasat School of Engineering, Thammasat University（工程学院，泰国朱拉安高校工程学院，泰国朱拉安大学）；Department of Computer Engineering and Digital Technology, Faculty of Engineering, Chulalongkorn University（计算机工程与数字技术系，工程学院，朱拉安大学）；Artificial Intelligence Association of Thailand（泰国人工智能协会）；School of Biomedical Engineering & Imaging Sciences, King’s College London（生物医学工程与成像科学学院，伦敦国王学院）；Siriraj Informatics and Data Innovation Center (SIData+), Faculty of Medicine, Siriraj Hospital, Mahidol University（Siriraj信息与数据创新中心（SIData+），医学学院，Siriraj医院，玛希道大学）
本文提出Beyond One World基准，通过30位英雄、90个正典版本及正典事件与道德困境任务，评估LLM跨多元宇宙角色一致性，并以Think-Act Matching衡量推理与行动对齐。
Integrating knowledge graphs and multilingual scholarly corpora for domain-adaptive LLMs in SSH
Huma-Num, CNRS（人文数字实验室，法国国家科学研究中心）；CNR - Institute for Computational Linguistics “A. Zampolli” (CNR-ILC)（意大利国家研究委员会 - 计算语言学“ A. 赞波利”研究所（CNR-ILC））；Inria（法国国家信息与自动化研究所）；Inist, CNRS（法国国家科学研究中心人文社会科学信息研究所）
A benchmark multimodal oro-dental dataset for large vision-language models
Suzhou Doctor Dental Clinic Co. Ltd（苏州博士口腔诊所）；South China University of Technology（华南理工大学）；Guangdong CAS Angels Biotechnology Co. Ltd（广东中科天使生物科技有限公司）；Guangdong Janus Biotechnology Co. Ltd（广东杰纳斯生物科技有限公司）；Zhejiang CAS Angels Biotechnology Co. Ltd（浙江中科天使生物科技有限公司）；OMRON SINIC X Corp.（欧姆龙先端技术研究所有限公司）；MingZheng Dental Clinic（明正牙科诊所）；University of Engineering & Technology, Peshawar（白沙瓦工程技术大学）
WorldView-Bench: A Benchmark for Evaluating Global Cultural Perspectives in Large Language Models
Information Technology University Department of Computer Science（信息科技大学计算机科学系）；Zayed University College of Interdisciplinary Studies（扎耶德大学跨学科研究学院）；Computer Science and Engineering Department（计算机科学与工程系）
