In an effort to regulate Machine Learning-driven (ML) systems, current auditing processes mostly focus on detecting harmful algorithmic biases. While these strategies have proven to be impactful, some values outlined in documents dealing with ethics in ML-driven systems are still underrepresented in auditing processes. Such unaddressed values mainly deal with contextual factors that cannot be easily quantified. In this paper, we develop a value-based assessment framework that is not limited to bias auditing and that covers prominent ethical principles for algorithmic systems. Our framework presents a circular arrangement of values with two bipolar dimensions that make common motivations and potential tensions explicit. In order to operationalize these high-level principles, values are then broken down into specific criteria and their manifestations. However, some of these value-specific criteria are mutually exclusive and require negotiation. As opposed to some other auditing frameworks that merely rely on ML researchers' and practitioners' input, we argue that it is necessary to include stakeholders that present diverse standpoints to systematically negotiate and consolidate value and criteria tensions. To that end, we map stakeholders with different insight needs, and assign tailored means for communicating value manifestations to them. We, therefore, contribute to current ML auditing practices with an assessment framework that visualizes closeness and tensions between values and we give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Partnership on AI（人工智能合作组织）；University of Cambridge（剑桥大学）；IBM Research（IBM研究院）；Carnegie Mellon University（卡内基梅隆大学）；Element AI；Intel Labs（英特尔实验室）；New York University（纽约大学）；The Alan Turing Institute（艾伦·图灵研究所）；Sony AI（索尼AI）
Learning the structure of Bayesian Networks: A quantitative assessment of the effect of different algorithmic schemes
Universitá degli Studi di Milano-Bicocca（米兰比可卡大学）；NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa（里斯本大学学院NOVA信息管理学院）；INESC Coimbra（科英布拉系统与计算机工程研究所）；University of Coimbra（科英布拉大学）；Stanford University（斯坦福大学）
