Social norms -- the unspoken commonsense rules about acceptable social behavior -- are crucial in understanding the underlying causes and intents of people's actions in narratives. For example, underlying an action such as "wanting to call cops on my neighbors" are social norms that inform our conduct, such as "It is expected that you report crimes."
We present Social Chemistry, a new conceptual formalism to study people's everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language. We introduce Social-Chem-101, a large-scale corpus that catalogs 292k rules-of-thumb such as "it is rude to run a blender at 5am" as the basic conceptual units. Each rule-of-thumb is further broken down with 12 different dimensions of people's judgments, including social judgments of good and bad, moral foundations, expected cultural pressure, and assumed legality, which together amount to over 4.5 million annotations of categorical labels and free-text descriptions.
Comprehensive empirical results based on state-of-the-art neural models demonstrate that computational modeling of social norms is a promising research direction. Our model framework, Neural Norm Transformer, learns and generalizes Social-Chem-101 to successfully reason about previously unseen situations, generating relevant (and potentially novel) attribute-aware social rules-of-thumb.
Probabilistic Aggregation and Targeted Embedding Optimization for Collective Moral Reasoning in Large Language Models
School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；School of Social Sciences and Technology, Technical University of Munich（社会科学与技术学院，慕尼黑技术大学）
University of Edinburgh（爱丁堡大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Washington（华盛顿大学）
In social settings, much of human behavior is governed by unspoken rules of conduct. For artificial systems to be fully integrated into social environments, adherence to such norms is a central prerequisite. We investigate whether contemporary NLG models can function as behavioral priors for systems deployed in social settings by generating action hypotheses that achieve predefined goals under moral constraints. Moreover, we examine if models can anticipate likely consequences of (im)moral actions, or explain why certain actions are preferable by generating relevant norms. For this purpose, we…
When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；Max Planck Institute for Intelligent Systems, Tübingen（图宾根人工智能研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
Exploring the psychology of LLMs' Moral and Legal Reasoning
INSPER Institute of Education and Research（INSPER 教育与研究学院）；FGV Direito Rio（FGV 里约法学院）；PUC-Rio（里约热内卢天主教大学）；Center for Humans and Machines, Max Planck Institute for Human Development（马克斯·普朗克人类发展研究所人类与机器中心）；Ruhr-University Bochum（波鸿鲁尔大学）；Federal University of Rio de Janeiro（里约热内卢联邦大学）；State University of Rio de Janeiro（里约热内卢州立大学）
Histoires Morales: A French Dataset for Assessing Moral Alignment
Laboratoire Hubert Curien（于贝尔·屈里安实验室）；CNRS（法国国家科学研究中心）；Université Lumière Lyon 2（里昂第二大学）；Université Claude Bernard Lyon 1（里昂第一大学）；ERIC（信息、表示与智能认知研究室）；Télécom Paris（巴黎高等电信学院）；Institut Polytechnique de Paris（巴黎理工学院）
MPI（马克斯·普朗克研究所）；ETH Zürich（苏黎世联邦理工学院）；MIT（麻省理工学院）；Harvard（哈佛大学）；IIT Kharagpur（印度理工学院卡拉格普尔分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）
AI systems are becoming increasingly intertwined with human life. In order to effectively collaborate with humans and ensure safety, AI systems need to be able to understand, interpret and predict human moral judgments and decisions. Human moral judgments are often guided by rules, but not always. A central challenge for AI safety is capturing the flexibility of the human moral mind -- the ability to determine when a rule should be broken, especially in novel or unusual situations. In this paper, we present a novel challenge set consisting of rule-breaking question answering (RBQA) of cases th…
Developing moral awareness in intelligent systems has shifted from a topic of philosophical inquiry to a critical and practical issue in artificial intelligence over the past decades. However, automated inference of everyday moral situations remains an under-explored problem. We present a text-based approach that predicts people's intuitive judgment of moral vignettes. Our methodology builds on recent work in contextualized language models and textual inference of moral sentiment. We show that a contextualized representation offers a substantial advantage over alternative representations based…
The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Media Synthesis and Forensics Lab（媒体合成与取证实验室）；Zhongguancun Laboratory（中关村实验室）
We present what we call the Interpretation Problem, whereby any rule in symbolic form is open to infinite interpretation in ways that we might disapprove of and argue that any attempt to build morality into machines is subject to it. We show how the Interpretation Problem in Artificial Intelligence is an illustration of Wittgenstein's general claim that no rule can contain the criteria for its own application, and that the risks created by this problem escalate in proportion to the degree to which to machine is causally connected to the world, in what we call the Law of Interpretative Exposure…
Georgia Institute of Technology（佐治亚理工学院）；Meta AI Research（Meta人工智能研究院）
Conversational agents have come increasingly closer to human competence in open-domain dialogue settings; however, such models can reflect insensitive, hurtful, or entirely incoherent viewpoints that erode a user's trust in the moral integrity of the system. Moral deviations are difficult to mitigate because moral judgments are not universal, and there may be multiple competing judgments that apply to a situation simultaneously. In this work, we introduce a new resource, not to authoritatively resolve moral ambiguities, but instead to facilitate systematic understanding of the intuitions, valu…
What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；York University（约克大学）
As AI systems become increasingly powerful and pervasive, there are growing concerns about machines' morality or a lack thereof. Yet, teaching morality to machines is a formidable task, as morality remains among the most intensely debated questions in humanity, let alone for AI. Existing AI systems deployed to millions of users, however, are already making decisions loaded with moral implications, which poses a seemingly impossible challenge: teaching machines moral sense, while humanity continues to grapple with it. To explore this challenge, we introduce Delphi, an experimental framework bas…
Social Catalysts, Not Moral Agents: The Illusion of Alignment in LLM Societies
Institute of Neuroscience, Chinese Academy of Sciences（中国科学院神经科学研究所）；School of Philosophy, Anhui University（安徽大学哲学学院）；Department of Psychology and Behavioral Sciences, Zhejiang University（浙江大学心理学与行为科学系）；Mental Health Education Center, North China Electric Power University（华北电力大学心理健康教育中心）
