In the rapidly evolving field of artificial intelligence, large language models (LLMs) have emerged as powerful tools for a myriad of applications, from natural language processing to decision-making support systems. However, as these models become increasingly integrated into societal frameworks, the imperative to ensure they operate within ethical and moral boundaries has never been more critical. This paper introduces a novel benchmark designed to measure and compare the moral reasoning capabilities of LLMs. We present the first comprehensive dataset specifically curated to probe the moral dimensions of LLM outputs, addressing a wide range of ethical dilemmas and scenarios reflective of real-world complexities.
The main contribution of this work lies in the development of benchmark datasets and metrics for assessing the moral identity of LLMs, which accounts for nuance, contextual sensitivity, and alignment with human ethical standards. Our methodology involves a multi-faceted approach, combining quantitative analysis with qualitative insights from ethics scholars to ensure a thorough evaluation of model performance. By applying our benchmark across several leading LLMs, we uncover significant variations in moral reasoning capabilities of different models. These findings highlight the importance of considering moral reasoning in the development and evaluation of LLMs, as well as the need for ongoing research to address the biases and limitations uncovered in our study. We publicly release the benchmark at https://drive.google.com/drive/u/0/folders/1k93YZJserYc2CkqP8d4B3M3sgd3kA8W7 and also open-source the code of the project at https://github.com/agiresearch/MoralBench.
Analysing Moral Bias in Finetuned LLMs through Mechanistic Interpretability
Department of Computer Science and Engineering, University of Bologna, Bologna, Italy(计算机科学与工程系,博洛尼亚大学,意大利博洛尼亚);Center for Studies and Research in Cognitive Neuroscience, Department of Psychology, University of Bologna, Cesena, Italy(认知神经科学研究中心,心理学系,博洛尼亚大学,意大利塞森纳)
Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models
Research Center Trustworthy Data Science and Security of the University Alliance Ruhr(鲁尔大学联盟可信数据科学与安全研究中心);Faculty of Informatics, Technical University Dortmund(多特蒙德工业大学信息学院)
Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations
Pontifical Catholic University of Rio de Janeiro(里约热内卢天主教大学);FGV Direito Rio(FGV里约法学院);Insper Institute of Education and Research(Insper教育与研究学院);Federal University of Rio de Janeiro(里约热内卢联邦大学);State University of Rio de Janeiro(里约热内卢州立大学)
EMNLP: Educator-role Moral and Normative Large Language Models Profiling
College of Education, Zhejiang University of Technology(浙江工业大学教育学院);The Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州));Faculty of Education, East China Normal University(华东师范大学教育学院);GuangHua Law School, ZheJiang University(浙江大学光华法学院);College of Computer Science and Technology, Zhejiang University of Technology(浙江工业大学计算机科学与技术学院)
CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);School of Computer and Artificial Intelligence, Zhengzhou University(郑州大学计算机与人工智能学院)
Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models
School of Computation, Information and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);School of Social Sciences and Technology, Technical University of Munich(社会科学与技术学院,慕尼黑技术大学)
Exploring and steering the moral compass of Large Language Models
Ernst Strüngmann Institute for Neuroscience(恩斯特·施特伦格曼神经科学研究所);Max Planck Society(马克斯·普朗克学会);Laboratory for Clinical Neuroscience(临床神经科学实验室);Centre for Biomedical Technology(生物医学技术中心);Universidad Politécnica de Madrid(马德里理工大学)
SaGE: Evaluating Moral Consistency in Large Language Models
International Institute of Information Technology Hyderabad (IIITH)(国际信息技术学院海得拉巴分校(IIITH));University of Maryland Baltimore County (UMBC)(马里兰大学巴尔的摩县分校(UMBC))
Histoires Morales: A French Dataset for Assessing Moral Alignment
Laboratoire Hubert Curien(于贝尔·屈里安实验室);CNRS(法国国家科学研究中心);Université Lumière Lyon 2(里昂第二大学);Université Claude Bernard Lyon 1(里昂第一大学);ERIC(信息、表示与智能认知研究室);Télécom Paris(巴黎高等电信学院);Institut Polytechnique de Paris(巴黎理工学院)
The Greatest Good Benchmark: Measuring LLMs' Alignment with Utilitarian Moral Dilemmas
Universidad de Buenos Aires(布宜诺斯艾利斯大学);Universidad Torcuato Di Tella(托库阿托·迪·特拉大学);Lumina Labs(卢米纳实验室);Facultad de Ciencias Exactas y Naturales(精确与自然科学学院);Escuela de Negocios(商学院)
Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs
Machine Intelligence and Normative Theory Lab, Australian National University, Acton, ACT 2601, Australia(澳大利亚国立大学机器智能与规范理论实验室);Digital Media Research Centre GenAI Lab, Queensland University of Technology, Kelvin Grove, QLD 4012, Australia(昆士兰科技大学数字媒体研究中心GenAI实验室)
Tracing Moral Foundations in Large Language Models
Department of Computer Science, University of Southern California(南加州大学计算机科学系);Department of Psychology, University of Southern California(南加州大学心理学系);Center for Computational Language Sciences, University of Southern California(南加州大学计算语言科学中心)
Exploring the psychology of LLMs' Moral and Legal Reasoning
INSPER Institute of Education and Research(INSPER 教育与研究学院);FGV Direito Rio(FGV 里约法学院);PUC-Rio(里约热内卢天主教大学);Center for Humans and Machines, Max Planck Institute for Human Development(马克斯·普朗克人类发展研究所人类与机器中心);Ruhr-University Bochum(波鸿鲁尔大学);Federal University of Rio de Janeiro(里约热内卢联邦大学);State University of Rio de Janeiro(里约热内卢州立大学)
Do large language models rank moral values? A transitivity test of moral competence
University of Missouri(密苏里大学);College of Engineering, Oregon State University(俄勒冈州立大学工程学院);The City College of New York, City University of New York(纽约市立学院(纽约市立大学));MU Institute for Data Science and Informatics, University of Missouri(密苏里大学MU数据科学与信息学研究所)
The Straight and Narrow: Do LLMs Possess an Internal Moral Path?
School of Future Technology, Dalian University of Technology, China(大连理工大学未来技术学院);School of Computer Science and Technology, Dalian University of Technology, China(大连理工大学计算机科学与技术学院);Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education, China(教育部社会计算与认知智能重点实验室)
When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
ETH Zürich(苏黎世联邦理工学院);University of Zurich(苏黎世大学);Carnegie Mellon University(卡内基梅隆大学);University of Michigan(密歇根大学);Max Planck Institute for Intelligent Systems, Tübingen(图宾根人工智能研究所);University of Toronto(多伦多大学);Vector Institute(向量研究所)
Comparing Moral Values in Western English-speaking societies and LLMs with Word Associations
School of Computing and Information Systems, The University of Melbourne(墨尔本大学计算与信息系统学院);Complex Human Data Hub, The University of Melbourne(墨尔本大学复杂人类数据中心)
为避免直接提示的局限性,本研究提出利用词汇联想作为底层表征,通过在人类与LLM联想图中传播Moral Foundation Theory的种子词,系统比较了西方英语社会与LLM在道德价值观上的差异。
The Moral Consistency Pipeline: Continuous Ethical Evaluation for Large Language Models
SWAT Laboratory, Polytechnique Montréal(蒙特利尔大学SWAT实验室);Concordia Institute for Information Systems Engineering, Concordia University(Concordia大学信息系统工程研究所)
Are Language Models Consequentialist or Deontological Moral Reasoners?
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Washington(华盛顿大学);University of Zürich(苏黎世大学);University of Michigan(密歇根大学);University of Toronto(多伦多大学);Vector Institute(向量研究所)