Responsible AI is becoming critical as AI is widely used in our everyday lives. Many companies that deploy AI publicly state that when training a model, we not only need to improve its accuracy, but also need to guarantee that the model does not discriminate against users (fairness), is resilient to noisy or poisoned data (robustness), is explainable, and more. In addition, these objectives are not only relevant to model training, but to all steps of end-to-end machine learning, which include data collection, data cleaning and validation, model training, model evaluation, and model management and serving. Finally, responsible AI is conceptually challenging, and supporting all the objectives must be as easy as possible. We thus propose three key research directions towards this vision - depth, breadth, and usability - to measure progress and introduce our ongoing research. First, responsible AI must be deeply supported where multiple objectives like fairness and robust must be handled together. To this end, we propose FR-Train, a holistic framework for fair and robust model training in the presence of data bias and poisoning. Second, responsible AI must be broadly supported, preferably in all steps of machine learning. Currently we focus on the data pre-processing steps and propose Slice Tuner, a selective data acquisition framework for training fair and accurate models, and MLClean, a data cleaning framework that also improves fairness and robustness. Finally, responsible AI must be usable where the techniques must be easy to deploy and actionable. We propose FairBatch, a batch selection approach for fairness that is effective and simple to use, and Slice Finder, a model evaluation tool that automatically finds problematic slices. We believe we scratched the surface of responsible AI for end-to-end machine learning and suggest research challenges moving forward.
Open Datasheets: Machine-readable Documentation for Open Datasets and Responsible AI Assessments
Microsoft（微软公司）；Microsoft, AI For Good Research Lab（微软公司AI向善研究实验室）；Microsoft, Research（微软研究院）；Microsoft, Office of Responsible AI（微软公司负责任AI办公室）；Microsoft, Open Innovation（微软公司开放创新部门）
Organizations of all sizes, across all industries and domains are leveraging artificial intelligence (AI) technologies to solve some of their biggest challenges around operations, customer experience, and much more. However, due to the probabilistic nature of AI, the risks associated with it are far greater than traditional technologies. Research has shown that these risks can range anywhere from regulatory, compliance, reputational, and user trust, to financial and even societal risks. Depending on the nature and size of the organization, AI technologies can pose a significant risk, if not us…
Explainable machine learning (ML) enables human learning from ML, human appeal of automated model decisions, regulatory compliance, and security audits of ML models. Explainable ML (i.e. explainable artificial intelligence or XAI) has been implemented in numerous open source and commercial packages and explainable ML is also an important, mandatory, or embedded aspect of commercial predictive modeling in industries like financial services. However, like many technologies, explainable ML can be misused, particularly as a faulty safeguard for harmful black-boxes, e.g. fairwashing or scaffolding,…
The recent release of ChatGPT has gained huge attention and discussion worldwide, with responsible AI being a key topic of discussion. How can we ensure that AI systems, including ChatGPT, are developed and adopted in a responsible way? To tackle the responsible AI challenges, various ethical principles have been released by governments, organisations, and companies. However, those principles are very abstract and not practical enough. Further, significant efforts have been put on algorithm-level solutions that only address a narrow set of principles, such as fairness and privacy. To fill the …
A Standardized Machine-readable Dataset Documentation Format for Responsible AI
Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；NASA IMPACT（美国国家航空航天局IMPACT）；Google（谷歌）；OpenML；Bayer（拜耳）；Dotphoton；NASA（美国国家航空航天局）；Meta；Oak Ridge National Laboratory（橡树岭国家实验室）；Plaixus Ltd（Plaixus有限公司）；North Carolina State University（北卡罗来纳州立大学）；Graphcore
Responsible AI Adoption in the Public Sector: A Data-Centric Taxonomy of AI Adoption Challenges
University of Tartu（塔尔图大学）；University of Hradec Kralove（赫拉德茨-克拉洛韦大学）；UAEM（墨西哥州自治大学）；University of Twente（特文特大学）；TU Delft（代尔夫特理工大学）；University of Warsaw（华沙大学）；Fortinet（飞塔公司）
Arizona State University（亚利桑那州立大学）；IBM Research – Thomas J. Watson Research Center（IBM研究院——托马斯·J·沃森研究中心）
In the current era, people and society have grown increasingly reliant on artificial intelligence (AI) technologies. AI has the potential to drive us towards a future in which all of humanity flourishes. It also comes with substantial risks for oppression and calamity. Discussions about whether we should (re)trust AI have repeatedly emerged in recent years and in many quarters, including industry, academia, healthcare, services, and so on. Technologists and AI researchers have a responsibility to develop trustworthy AI systems. They have responded with great effort to design more responsible A…
Responsible Artificial Intelligence (AI) - the practice of developing, evaluating, and maintaining accurate AI systems that also exhibit essential properties such as robustness and explainability - represents a multifaceted challenge that often stretches standard machine learning tooling, frameworks, and testing methods beyond their limits. In this paper, we present two new software libraries - hydra-zen and the rAI-toolbox - that address critical needs for responsible AI engineering. hydra-zen dramatically simplifies the process of making complex AI applications configurable, and their behavi…
In the last few years, AI continues demonstrating its positive impact on society while sometimes with ethically questionable consequences. Building and maintaining public trust in AI has been identified as the key to successful and sustainable innovation. This chapter discusses the challenges related to operationalizing ethical AI principles and presents an integrated view that covers high-level ethical AI principles, the general notion of trust/trustworthiness, and product/process support in the context of responsible AI, which helps improve both trust and trustworthiness of AI for a wider se…
Although AI is transforming the world, there are serious concerns about its ability to behave and make decisions responsibly. Many ethical regulations, principles, and frameworks for responsible AI have been issued recently. However, they are high level and difficult to put into practice. On the other hand, most AI researchers focus on algorithmic solutions, while the responsible AI challenges actually crosscut the entire engineering lifecycle and components of AI systems. To close the gap in operationalizing responsible AI, this paper aims to develop a roadmap on software engineering for resp…
Requirements Engineering (RE) is the discipline for identifying, analyzing, as well as ensuring the implementation and delivery of user, technical, and societal requirements. Recently reported issues concerning the acceptance of Artificial Intelligence (AI) solutions after deployment, e.g. in the medical, automotive, or scientific domains, stress the importance of RE for designing and delivering Responsible AI systems. In this paper, we argue that RE should not only be carefully conducted but also tailored for Responsible AI. We outline related challenges for research and practice.
Indian Institute of Technology, Hyderabad（印度理工学院海得拉巴分校）；CVR College of Engineering（CVR工程学院）；Pandit Deendayal Energy University（潘迪特·丁达亚尔能源大学）
The advances in Artificial Intelligence are creating new opportunities to improve lives of people around the world, from business to healthcare, from lifestyle to education. For example, some systems profile the users using their demographic and behavioral characteristics to make certain domain-specific predictions. Often, such predictions impact the life of the user directly or indirectly (e.g., loan disbursement, determining insurance coverage, shortlisting applications, etc.). As a result, the concerns over such AI-enabled systems are also increasing. To address these concerns, such systems…
There have been increasing concerns about Artificial Intelligence (AI) due to its unfathomable potential power. To make AI address ethical challenges and shun undesirable outcomes, researchers proposed to develop socially responsible AI (SRAI). One of these approaches is causal learning (CL). We survey state-of-the-art methods of CL for SRAI. We begin by examining the seven CL tools to enhance the social responsibility of AI, then review how existing works have succeeded using these tools to tackle issues in developing SRAI such as fairness. The goal of this survey is to bring forefront the po…
TECNALIA；ENSTA, Institute Polytechnique Paris（巴黎综合理工学院ENSTA）；INRIA（法国国家信息与自动化研究所）；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））；Basque Center for Applied Mathematics (BCAM)（巴斯克应用数学中心（BCAM））；Segula Technologies（Segula科技公司）；Sorbonne Universitè（索邦大学）；DaSCI Andalusian Institute of Data Science and Computational Intelligence, University of Granada（格拉纳达大学DaSCI安达卢西亚数据科学与计算智能研究所）；Telefonica（西班牙电信公司）
In the last years, Artificial Intelligence (AI) has achieved a notable momentum that may deliver the best of expectations over many application sectors across the field. For this to occur, the entire community stands in front of the barrier of explainability, an inherent problem of AI techniques brought by sub-symbolism (e.g. ensembles or Deep Neural Networks) that were not present in the last hype of AI. Paradigms underlying this problem fall within the so-called eXplainable AI (XAI) field, which is acknowledged as a crucial feature for the practical deployment of AI models. This overview exa…
Although artificial intelligence (AI) is solving real-world challenges and transforming industries, there are serious concerns about its ability to behave and make decisions in a responsible way. Many AI ethics principles and guidelines for responsible AI have been recently issued by governments, organisations, and enterprises. However, these AI ethics principles and guidelines are typically high-level and do not provide concrete guidance on how to design and develop responsible AI systems. To address this shortcoming, we first present an empirical study where we interviewed 21 scientists and …
For AI technology to fulfill its full promises, we must have effective means to ensure Responsible AI behavior and curtail potential irresponsible use, e.g., in areas of privacy protection, human autonomy, robustness, and prevention of biases and discrimination in automated decision making. Recent literature in the field has identified serious shortcomings of narrow technology focused and formalism-oriented research and has proposed an interdisciplinary approach that brings the social context into the scope of study. In this paper, we take a sociotechnical approach to propose a more expansive …
University of Edinburgh（爱丁堡大学）；Alan Turing Institute（艾伦·图灵研究所）
Artificial Intelligence (AI) provides many opportunities to improve private and public life. Discovering patterns and structures in large troves of data in an automated manner is a core component of data science, and currently drives applications in computational biology, finance, law and robotics. However, such a highly positive impact is coupled with significant challenges: How do we understand the decisions suggested by these systems in order that we can trust them? How can they be held accountable for those decisions? In this short survey, we cover some of the motivations and trends in the…
Legislation and public sentiment throughout the world have promoted fairness metrics, explainability, and interpretability as prescriptions for the responsible development of ethical artificial intelligence systems. Despite the importance of these three pillars in the foundation of the field, they can be challenging to operationalize and attempts to solve the problems in production environments often feel Sisyphean. This difficulty stems from a number of factors: fairness metrics are computationally difficult to incorporate into training and rarely alleviate all of the harms perpetrated by the…
Microsoft（微软）；University of Michigan, School of Information（密歇根大学信息学院）
Data is central to the development and evaluation of machine learning (ML) models. However, the use of problematic or inappropriate datasets can result in harms when the resulting models are deployed. To encourage responsible AI practice through more deliberate reflection on datasets and transparency around the processes by which they are created, researchers and practitioners have begun to advocate for increased data documentation and have proposed several data documentation frameworks. However, there is little research on whether these data documentation frameworks meet the needs of ML pract…
A Framework for Responsible AI Systems: Building Societal Trust through Domain Definition, Trustworthy AI Design, Auditability, Accountability, and Governance
Deparment of Computer Science and Artificial Intelligence and Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI), University of Granada（计算机科学与人工智能系和安达卢西亚数据科学与计算智能研究所（DaSCI），格拉纳达大学）；TECNALIA (BRTA)（TECNALIA（BRTA））；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））；School of Engineering, Cornell University（工程学院，康奈尔大学）；ADIA Lab（ADIA实验室）；Computational Research Department, Lawrence Berkeley National Laboratory（计算研究部，劳伦斯伯克利国家实验室）；Intelligent Systems for Robotics and Automation Laboratory, Macau University of Science and Technology（机器人与自动化智能系统实验室，澳门科学技术大学）
The impact of Artificial Intelligence does not depend only on fundamental research and technological developments, but for a large part on how these systems are introduced into society and used in everyday situations. AI is changing the way we work, live and solve challenges but concerns about fairness, transparency or privacy are also growing. Ensuring responsible, ethical AI is more than designing systems whose result can be trusted. It is about the way we design them, why we design them, and who is involved in designing them. In order to develop and use AI responsibly, we need to work towar…
Google Research（谷歌研究院）；Hugging Face（拥抱脸）；DAIR（分布式人工智能研究所）；Deepmind（深度思维）
Research on fairness, accountability, transparency and ethics of AI-based interventions in society has gained much-needed momentum in recent years. However it lacks an explicit alignment with a set of normative values and principles that guide this research and interventions. Rather, an implicit consensus is often assumed to hold for the values we impart into our models - something that is at odds with the pluralistic world we live in. In this paper, we put forth the doctrine of universal human rights as a set of globally salient and cross-culturally recognized set of values that can serve as …
Challenges for Responsible AI Design and Workflow Integration in Healthcare: A Case Study of Automatic Feeding Tube Qualification in Radiology
Microsoft Health Futures（微软健康未来）；UCLH NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）；Canterbury Christ Church University（坎特伯雷基督教会大学）；University College London（伦敦大学学院）；Cambridge University Hospitals NHS Foundation Trust（剑桥大学医院NHS基金会信托）；Microsoft Research Cambridge（微软研究院剑桥）
Warsaw University of Technology（华沙理工大学）；Faculty of Mathematics and Information Science（数学与信息科学学院）；Samsung Research & Development Institute（三星研发研究院）
The increasing amount of available data, computing power, and the constant pursuit for higher performance results in the growing complexity of predictive models. Their black-box nature leads to opaqueness debt phenomenon inflicting increased risks of discrimination, lack of reproducibility, and deflated performance due to data drift. To manage these risks, good MLOps practices ask for better validation of model performance and fairness, higher explainability, and continuous monitoring. The necessity of deeper model transparency appears not only from scientific and social domains, but also emer…
Practitioners from diverse occupations and backgrounds are increasingly using machine learning (ML) methods. Nonetheless, studies on ML Practitioners typically draw populations from Big Tech and academia, as researchers have easier access to these communities. Through this selection bias, past research often excludes the broader, lesser-resourced ML community -- for example, practitioners working at startups, at non-tech companies, and in the public sector. These practitioners share many of the same ML development difficulties and ethical conundrums as their Big Tech counterparts; however, the…
The financial services industry has unique explainability and fairness challenges arising from compliance and ethical considerations in credit decisioning. These challenges complicate the use of model machine learning and artificial intelligence methods in business decision processes.
Responsible Machine Learning via Mixed-Integer Optimization
Center for Artificial Intelligence in Society（人工智能与社会中心）；Department of Computer Science（计算机科学系）；University of Southern California（南加州大学）；Department of Industrial & Systems Engineering（工业与系统工程系）
Global Fishing Watch（全球渔业观察）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Nautical Crime Investigation Services（海事犯罪调查服务处）
Many fishing vessels use forced labor, but identifying vessels that engage in this practice is challenging because few are regularly inspected. We developed a positive-unlabeled learning algorithm using vessel characteristics and movement patterns to estimate an upper bound of the number of positive cases of forced labor, with the goal of helping make accurate, responsible, and fair decisions. 89% of the reported cases of forced labor were correctly classified as positive (recall) while 98% of the vessels certified as having decent working conditions were correctly classified as negative. The …
The data revolution has generated a huge demand for data-driven solutions. This demand propels a growing number of easy-to-use tools and training for aspiring data scientists that enable the rapid building of predictive models. Today, weapons of math destruction can be easily built and deployed without detailed planning and validation. This rapidly extends the list of AI failures, i.e. deployments that lead to financial losses or even violate democratic values such as equality, freedom and justice. The lack of planning, rules and standards around the model development leads to the ,,anarchisat…
Technical University of Denmark（丹麦技术大学）；Universität zu Lübeck（吕贝克大学）；Kiel University（基尔大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；UniTransferKlinik Lübeck GmbH（吕贝克联合转移诊所有限公司）；Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering (IMTE)（弗劳恩霍夫个体化与细胞医学工程研究机构（IMTE））
Machine learning is expected to fuel significant improvements in medical care. To ensure that fundamental principles such as beneficence, respect for human autonomy, prevention of harm, justice, privacy, and transparency are respected, medical machine learning systems must be developed responsibly. Many high-level declarations of ethical principles have been put forth for this purpose, but there is a severe lack of technical guidelines explicating the practical consequences for medical machine learning. Similarly, there is currently considerable uncertainty regarding the exact regulatory requi…
University of Engineering and Technology, Vietnam National University（越南国家大学工程技术大学）；Umeå University（于默奥大学）；University of Sydney（悉尼大学）；ReML.AI - Reliable Machine Learning Lab, International（ReML.AI——国际可靠机器学习实验室）；Hanoi University of Science, Vietnam National University（越南国家大学河内科学大学）；University of Glasgow（格拉斯哥大学）
This paper reports on the ReINTEL Shared Task for Responsible Information Identification on social network sites, which is hosted at the seventh annual workshop on Vietnamese Language and Speech Processing (VLSP 2020). Given a piece of news with respective textual, visual content and metadata, participants are required to classify whether the news is `reliable' or `unreliable'. In order to generate a fair benchmark, we introduce a novel human-annotated dataset of over 10,000 news collected from a social network in Vietnam. All models will be evaluated in terms of AUC-ROC score, a typical evalu…
The AI-alignment problem arises when there is a discrepancy between the goals that a human designer specifies to an AI learner and a potential catastrophic outcome that does not reflect what the human designer really wants. We argue that a formalism of AI alignment that does not distinguish between strategic and agnostic misalignments is not useful, as it deems all technology as un-safe. We propose a definition of a strategic-AI-alignment and prove that most machine learning algorithms that are being used in practice today do not suffer from the strategic-AI-alignment problem. However, without…
University of Waterloo（滑铁卢大学）；Waterloo Artificial Intelligence Institute（滑铁卢人工智能研究所）；DarwinAI Corp.（DarwinAI公司）
In this study, we take a departure and explore an explainability-driven strategy to data auditing, where actionable insights into the data at hand are discovered through the eyes of quantitative explainability on the behaviour of a dummy model prototype when exposed to data. We demonstrate this strategy by auditing two popular medical benchmark datasets, and discover hidden data quality issues that lead deep learning models to make predictions for the wrong reasons. The actionable insights gained from this explainability driven data auditing strategy is then leveraged to address the discovered…
Twitter（推特）；Facebook（脸书）；SigOpt, an Intel company（SigOpt（英特尔公司旗下））；Valohai；Paradigm（第四范式）；ChaLearn
This paper presents the results and insights from the black-box optimization (BBO) challenge at NeurIPS 2020 which ran from July-October, 2020. The challenge emphasized the importance of evaluating derivative-free optimizers for tuning the hyperparameters of machine learning models. This was the first black-box optimization challenge with a machine learning emphasis. It was based on tuning (validation set) performance of standard machine learning models on real datasets. This competition has widespread impact as black-box optimization (e.g., Bayesian optimization) is relevant for hyperparamete…
Designing Interpretable ML System to Enhance Trust in Healthcare: A Systematic Review to Proposed Responsible Clinician-AI-Collaboration Framework
Grado Department of Industrial & Systems Engineering, Virginia Tech（弗吉尼亚理工大学格拉多工业与系统工程系）；Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University（迪肯大学智能系统研究与创新研究所）；School of Mathematics, Physics, and Computing, University of Southern Queensland（南昆士兰大学数学、物理与计算学院）；Center for Health Research, University of Southern Queensland（南昆士兰大学健康研究中心）
With the agreement of my coauthors, I Zhangyang Wang would like to withdraw the manuscript "Stacked Approximated Regression Machine: A Simple Deep Learning Approach". Some experimental procedures were not included in the manuscript, which makes a part of important claims not meaningful. In the relevant research, I was solely responsible for carrying out the experiments; the other coauthors joined in the discussions leading to the main algorithm. Please see the updated text for more details.
