We are moving towards a future where Artificial Intelligence (AI) based agents make many decisions on behalf of humans. From healthcare decision making to social media censoring, these agents face problems, and make decisions with ethical and societal implications. Ethical behaviour is a critical characteristic that we would like in a human-centric AI. A common observation in human-centric industries, like the service industry and healthcare, is that their professionals tend to break rules, if necessary, for pro-social reasons. This behaviour among humans is defined as pro-social rule breaking. To make AI agents more human centric, we argue that there is a need for a mechanism that helps AI agents identify when to break rules set by their designers. To understand when AI agents need to break rules, we examine the conditions under which humans break rules for pro-social reasons. In this paper, we present a study that introduces a 'vaccination strategy dilemma' to human participants and analyses their responses. In this dilemma, one needs to decide whether they would distribute Covid-19 vaccines only to members of a high-risk group (follow the enforced rule) or, in selected cases, administer the vaccine to a few social influencers (break the rule), which might yield an overall greater benefit to society. The results of the empirical study suggest a relationship between stakeholder utilities and pro-social rule breaking (PSRB), which neither deontological nor utilitarian ethics completely explain. Finally, the paper discusses the design characteristics of an ethical agent capable of PSRB and the future research directions on PSRB in the AI realm. We hope that this will inform the design of future AI agents, and their decision-making behaviour.
An ambitious goal for machine learning is to create agents that behave ethically: The capacity to abide by human moral norms would greatly expand the context in which autonomous agents could be practically and safely deployed, e.g. fully autonomous vehicles will encounter charged moral decisions that complicate their deployment. While ethical agents could be trained by rewarding correct behavior under a specific moral theory (e.g. utilitarianism), there remains widespread disagreement about the nature of morality. Acknowledging such disagreement, recent work in moral philosophy proposes that e…
Artificial reinforcement learning (RL) is a widely used technique in artificial intelligence that provides a general method for training agents to perform a wide variety of behaviours. RL as used in computer science has striking parallels to reward and punishment learning in animal and human brains. I argue that present-day artificial RL agents have a very small but nonzero degree of ethical importance. This is particularly plausible for views according to which sentience comes in degrees based on the abilities and complexities of minds, but even binary views on consciousness should assign non…
Reinforcement learning systems will to a greater and greater extent make decisions that significantly impact the well-being of humans, and it is therefore essential that these systems make decisions that conform to our expectations of morally good behavior. The morally good is often defined in causal terms, as in whether one's actions have in fact caused a particular outcome, and whether the outcome could have been anticipated. We propose an online reinforcement learning method that learns a policy under the constraint that the agent should not be the cause of harm. This is accomplished by def…
Building Altruistic and Moral AI Agent with Brain-inspired Emotional Empathy Mechanisms
Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China（脑启发认知人工智能实验室，自动化研究所，中国科学院，北京，中国）；Beijing Key Laboratory of Safe AI and Superalignment, China（北京安全人工智能与超对齐关键实验室，中国）；Beijing Institute of AI Safety and Governance, China（北京人工智能安全与治理研究所，中国）；Long-term AI, Beijing, China（长期人工智能，北京，中国）；School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（人工智能学院，中国科学院大学，北京，中国）；School of Humanities, University of Chinese Academy of Sciences, Beijing, China（人文学院，中国科学院大学，北京，中国）；State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences, Shanghai, China（脑认知与脑启发智能技术国家重点实验室，中国科学院，上海，中国）
Computer Science Department, University of Cape Town（开普敦大学计算机科学系）；University of Cape Town（开普敦大学）
Artificial Moral Agents (AMA's) is a field in computer science with the purpose of creating autonomous machines that can make moral decisions akin to how humans do. Researchers have proposed theoretical means of creating such machines, while philosophers have made arguments as to how these machines ought to behave, or whether they should even exist. Of the currently theorised AMA's, all research and design has been done with either none or at most one specified normative ethical theory as basis. This is problematic because it narrows down the AMA's functional ability and versatility which in t…
In this meta-ethnography, we explore three different angles of ethical artificial intelligence (AI) design implementation including the philosophical ethical viewpoint, the technical perspective, and framing through a political lens. Our qualitative research includes a literature review that highlights the cross-referencing of these angles by discussing the value and drawbacks of contrastive top-down, bottom-up, and hybrid approaches previously published. The novel contribution to this framework is the political angle, which constitutes ethics in AI either being determined by corporations and …
As we grant artificial intelligence increasing power and independence in contexts like healthcare, policing, and driving, AI faces moral dilemmas but lacks the tools to solve them. Warnings from regulators, philosophers, and computer scientists about the dangers of unethical artificial intelligence have spurred interest in automated ethics-i.e., the development of machines that can perform ethical reasoning. However, prior work in automated ethics rarely engages with philosophical literature. Philosophers have spent centuries debating moral dilemmas so automated ethics will be most nuanced, co…
Much of the existing research on the social and ethical impact of Artificial Intelligence has been focused on defining ethical principles and guidelines surrounding Machine Learning (ML) and other Artificial Intelligence (AI) algorithms [IEEE, 2017, Jobin et al., 2019]. While this is extremely useful for helping define the appropriate social norms of AI, we believe that it is equally important to discuss both the potential and risks of ML and to inspire the community to use ML for beneficial objectives. In the present article, which is specifically aimed at ML practitioners, we thus focus more…
Informed AI Regulation: Comparing the Ethical Frameworks of Leading LLM Chatbots Using an Ethics-Based Audit to Assess Moral Reasoning and Normative Values
Charles Sturt University（查尔斯特大学）；University in Hartford（哈特福德大学）；University of Connecticut（康涅狄格大学）；Ohio Northern University（俄亥俄北方大学）
The field of machine ethics is concerned with the question of how to embed ethical behaviors, or a means to determine ethical behaviors, into artificial intelligence (AI) systems. The goal is to produce artificial moral agents (AMAs) that are either implicitly ethical (designed to avoid unethical consequences) or explicitly ethical (designed to behave ethically). Van Wynsberghe and Robbins' (2018) paper Critiquing the Reasons for Making Artificial Moral Agents critically addresses the reasons offered by machine ethicists for pursuing AMA research; this paper, co-authored by machine ethicists a…
Engaging Engineering Teams Through Moral Imagination: A Bottom-Up Approach for Responsible Innovation and Ethical Culture Change in Technology Companies
本文提出“道德想象力”方法论，通过在 Google 开展 50 余场工作坊，自下而上推动工程团队负责任创新与伦理文化变革，作为现有伦理审查与合规机制的关键补充。
Imperial College London（伦敦帝国理工学院）；National Institute for Health Research（国家健康研究所）
Artificial intelligence (AI) assisting with antimicrobial prescribing raises significant moral questions. Utilising ethical frameworks alongside AI-driven systems, while considering infection specific complexities, can support moral decision making to tackle antimicrobial resistance.
North Carolina State University（北卡罗来纳州立大学）；Oklahoma State University（俄克拉荷马州立大学）；University of Bristol（布里斯托大学）
Autonomous Vehicles (AVs) raise important social and ethical concerns, especially about accountability, dignity, and justice. We focus on the specific concerns arising from how AV technology will affect the lives and livelihoods of professional and semi-professional drivers. Whereas previous studies of such concerns have focused on the opinions of experts, we seek to understand these ethical and societal challenges from the perspectives of the drivers themselves. To this end, we adopted a qualitative research methodology based on semi-structured interviews. This is an established social scienc…
Institut Polytechnique de Paris（巴黎理工学院）；Télécom Paris（巴黎电信学院）；Minsait by Indra Sistemas SA（英德拉系统公司旗下Minsait公司）；Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS)（智能技术卓越研究中心（CiTIUS））；Universidade de Santiago de Compostela（圣地亚哥德孔波斯特拉大学）
Visualizations have a potentially enormous influence on how data are used to make decisions across all areas of human endeavor. However, it is not clear how this power connects to ethical duties: what obligations do we have when it comes to visualizations and visual analytics systems, beyond our duties as scientists and engineers? Drawing on historical and contemporary examples, I address the moral components of the design and use of visualizations, identify some ongoing areas of visualization research with ethical dilemmas, and propose a set of additional moral obligations that we have as des…
University of Kentucky（肯塔基大学）；University of Southern California（南加州大学）；University of Michigan（密歇根大学）；IBM T.J. Watson Research Center（IBM T.J. 沃森研究中心）；UNSW（新南威尔士大学）；Data61/CSIRO（澳大利亚联邦科学与工业研究组织 Data61）；TU Berlin（柏林工业大学）
The recent surge in interest in ethics in artificial intelligence may leave many educators wondering how to address moral, ethical, and philosophical issues in their AI courses. As instructors we want to develop curriculum that not only prepares students to be artificial intelligence practitioners, but also to understand the moral, ethical, and philosophical impacts that artificial intelligence will have on society. In this article we provide practical case studies and links to resources for use by AI educators. We also provide concrete suggestions on how to integrate AI ethics into a general …
The ethics of Machine Learning has become an unavoidable topic in the AI Community. The deployment of machine learning systems in multiple social contexts has resulted in a closer ethical scrutiny of the design, development, and application of these systems. The AI/ML community has come to terms with the imperative to think about the ethical implications of machine learning, not only as a product but also as a practice (Birhane, 2021; Shen et al. 2021). The critical question that is troubling many debates is what can constitute an ethically accountable machine learning system. In this paper we…
When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
ETH Zürich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；Max Planck Institute for Intelligent Systems, Tübingen（图宾根人工智能研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
Ethicists, policy-makers, and the general public have questioned whether artificial entities such as robots warrant rights or other forms of moral consideration. There is little synthesis of the research on this topic so far. We identify 294 relevant research or discussion items in our literature review of this topic. There is widespread agreement among scholars that some artificial entities could warrant moral consideration in the future, if not also the present. The reasoning varies, such as concern for the effects on artificial entities and concern for the effects on human society. Beyond t…
The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Media Synthesis and Forensics Lab（媒体合成与取证实验室）；Zhongguancun Laboratory（中关村实验室）
This paper presents the design process and empirical evaluation of a new tool for enabling ethics by design: The Moral-IT Cards. Better tools are needed to support the role of technologists in addressing ethical issues during system design. These physical cards support reflection by technologists on normative aspects of technology development, specifically on emerging risks, appropriate safeguards and challenges of implementing these in the system. We discuss how the cards were developed and tested within 5 workshops with 20 participants from both research and commercial settings. We consider …
