Research in Responsible AI has developed a range of principles and practices to ensure that machine learning systems are used in a manner that is ethical and aligned with human values. However, a critical yet often neglected aspect of ethical ML is the ethical implications that appear when designing evaluations of ML systems. For instance, teams may have to balance a trade-off between highly informative tests to ensure downstream product safety, with potential fairness harms inherent to the implemented testing procedures. We conceptualize ethics-related concerns in standard ML evaluation techniques. Specifically, we present a utility framework, characterizing the key trade-off in ethical evaluation as balancing information gain against potential ethical harms. The framework is then a tool for characterizing challenges teams face, and systematically disentangling competing considerations that teams seek to balance. Differentiating between different types of issues encountered in evaluation allows us to highlight best practices from analogous domains, such as clinical trials and automotive crash testing, which navigate these issues in ways that can offer inspiration to improve evaluation processes in ML. Our analysis underscores the critical need for development teams to deliberately assess and manage ethical complexities that arise during the evaluation of ML systems, and for the industry to move towards designing institutional policies to support ethical evaluations.
Ethical and Explainable AI in Reusable MLOps Pipelines
Washington University of Science and Technology（科学与技术华盛顿大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）；Karlstad University（卡尔斯塔德大学）；American University of Bahrain（巴林美国大学）
The Switch, the Ladder, and the Matrix: Models for Classifying AI Systems
Oxford Internet Institute, University of Oxford（牛津大学牛津互联网研究院）；R&D Data Office, AstraZeneca PLC（阿斯利康制药研发数据办公室）；Department of Statistical Science, University College London（伦敦大学学院统计科学系）；Department of Legal Studies, University of Bologna（博洛尼亚大学法律研究系）
Towards a Framework for Supporting the Ethical and Regulatory Certification of AI Systems
St. Pölten University of Applied Sciences（施普伦特应用科学大学）；Digital for Planet - D4P（数字为行星 - D4P）；University of Luxembourg（卢森堡大学）；Netcompany-Intrasoft S.A.（Netcompany-Intrasoft公司）
Massachusetts Institute of Technology（麻省理工学院）；Microsoft Research（微软研究院）；Stanford University（斯坦福大学）；Vector Institute（向量研究院）；New York University（纽约大学）；University of Toronto（多伦多大学）
Large Language Models as a Semantic Interface and Ethical Mediator in Neuro-Digital Ecosystems: Conceptual Foundations and a Regulatory Imperative
Immanuel Kant Baltic Federal University（伊曼努尔·康德波罗的海联邦大学）；Plekhanov Russian University of Economics（普列汉诺夫俄罗斯经济大学）；Pirogov National Medical and Surgical Center（皮罗戈夫国家医学外科中心）
Toward equipping Artificial Moral Agents with multiple ethical theories
Computer Science Department, University of Cape Town（开普敦大学计算机科学系）；University of Cape Town（开普敦大学）
人工道德代理（Artificial Moral Agents，AMA）是计算机科学中的一个领域，其目的是创建能够做出类似于人类行为的道德决策的自主机器。研究人员提出了创造这种机器的理论方法，而哲学家则对这些机器应该如何运作，或者它们是否应该存在进行了争论。在目前理论化的AMA中，所有的研究和设计都是在没有或至多有一个特定的规范伦理理论作为基础的情况下完成的。这是有问题的，因为它缩小了AMA的功能能力和多功能性，这反过来又导致了少数人同意的道德结果（从而破坏了AMA在人类意义上的道德能力）。作为解决方案，我们设计了一个三层模型的一般规范的道德理论，可用于序列化的人和企业的道德观点的AMA在推理过程中使用。四个具体的道德规范（康德主义，神的命令理论，功利主义和利己主义）进行了建模和评估的概念证明规范建模。此外，所有模型都被序列化为XML/XSD，作为支持计算机化的证明。
Development of management systems using artificial intelligence systems and machine learning methods for boards of directors (preprint, unofficial translation)
Moscow Institute of Physics and Technology（莫斯科物理技术学院）
Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers
Princeton University（普林斯顿大学）
机器学习数据集引发了对隐私、偏见和不道德应用的担忧，导致DukeMTMC、MS-Celeb-1 M和Tiny Images等知名数据集被撤回。作为回应，机器学习社区呼吁在数据集创建中制定更高的道德标准。为了帮助为这些工作提供信息，我们研究了三个有影响力但存在道德问题的人脸和人物识别数据集-- Labeled Faces in the Wild（LFW）、MS-Celeb-1 M和DukeMTM --通过分析近1000篇引用它们的论文。我们发现，衍生数据集和模型的创建、更广泛的技术和社会变革、许可证的不明确以及数据集管理实践可能会带来广泛的伦理问题。最后，我们提出了一个分布式的方法来减轻伤害，考虑整个生存周期的数据集。
