在使用人工智能（AI）进行高风险决策时，公平性正日益受到关注，但仅通过技术手段确保公平性具有挑战性：不存在普遍接受的公平性度量，公平性依赖于具体情境，而且对于何为公平可能存在相互冲突的观点。因此，让通常没有 AI 或公平性背景的利益相关者参与进来，是一条很有前景的途径。直接让利益相关者参与的研究仍处于初期阶段，在如何支持利益相关者就公平性提供反馈，以及如何将这种反馈整合到 AI 模型中，仍有许多问题。我们的工作采用一种方法：利益相关者可以针对具体决策实例及其结果的公平性给出反馈，随后重新训练 AI 模型。为研究该方法，我们对用于贷款申请的复杂信用评级 AI 模型开展了两项研究。在研究 1 中，我们收集了 58 名普通用户对贷款申请决策的反馈，并进行离线实验，以考察其对准确率和公平性指标的影响。在研究 2 中，我们向 66 名参与者展示其反馈在公平性方面的结果，从而深化这项调查，随后开展进一步的离线分析。我们的工作贡献了两个数据集及相关代码框架以推动后续研究，强调了利用普通用户反馈改善 AI 公平性的机遇与挑战，并讨论了开发更贴近利益相关者公平性观点的 AI 应用所具有的实践意义。
英文摘要：
Fairness is a growing concern for high-risk decision-making using Artificial Intelligence (AI) but ensuring it through purely technical means is challenging: there is no universally accepted fairness measure, fairness is context-dependent, and there might be conflicting perspectives on what is considered fair. Thus, involving stakeholders, often without a background in AI or fairness, is a promising avenue. Research to directly involve stakeholders is in its infancy, and many questions remain on how to support stakeholders to feedback on fairness, and how this feedback can be integrated into AI models. Our work follows an approach where stakeholders can give feedback on specific decision instances and their outcomes with respect to their fairness, and then to retrain an AI model. In order to investigate this approach, we conducted two studies of a complex AI model for credit rating used in loan applications. In study 1, we collected feedback from 58 lay users on loan application decisions, and conducted offline experiments to investigate the effects on accuracy and fairness metrics. In study 2, we deepened this investigation by showing 66 participants the results of their feedback with respect to fairness, and then conducted further offline analyses. Our work contributes two datasets and associated code frameworks to bootstrap further research, highlights the opportunities and challenges of employing lay user feedback for improving AI fairness, and discusses practical implications for developing AI applications that more closely reflect stakeholder views about fairness.
"I think this is fair": Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment
School of Computing Science University of Glasgow Glasgow United Kingdom（计算机科学学院格拉斯哥大学格拉斯哥英国）；Fujitsu Research Fujitsu Limited Kawasaki Japan（富士通研究富士通有限公司川崎日本）；University of Glasgow（格拉斯哥大学）；Fujitsu Limited（富士通有限公司）
EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders
School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；FUJITSU LIMITED（富士通有限公司）；Graduate School of Arts and Sciences, The University of Tokyo（东京大学文理研究生院）；Artificial Intelligence Laboratory, FUJITSU LIMITED（富士通有限公司人工智能实验室）
KAIST（韩国科学技术院）；Institute for Basic Science（基础科学研究所）
Responsible Artificial Intelligence (AI) proposes a framework that holds all stakeholders involved in the development of AI to be responsible for their systems. It, however, fails to accommodate the possibility of holding AI responsible per se, which could close some legal and moral gaps concerning the deployment of autonomous and self-learning systems. We discuss three notions of responsibility (i.e., blameworthiness, accountability, and liability) for all stakeholders, including AI, and suggest the roles of jurisdiction and the general public in this matter.
City, University of London（伦敦城市大学）；Fujitsu Laboratories of Europe（富士通欧洲实验室）；Intesa Sanpaolo S.p.A.（意大利联合圣保罗银行）
With Artificial intelligence (AI) to aid or automate decision-making advancing rapidly, a particular concern is its fairness. In order to create reliable, safe and trustworthy systems through human-centred artificial intelligence (HCAI) design, recent efforts have produced user interfaces (UIs) for AI experts to investigate the fairness of AI models. In this work, we provide a design space exploration that supports not only data scientists but also domain experts to investigate AI fairness. Using loan applications as an example, we held a series of workshops with loan officers and data scienti…
In the last few years, AI continues demonstrating its positive impact on society while sometimes with ethically questionable consequences. Building and maintaining public trust in AI has been identified as the key to successful and sustainable innovation. This chapter discusses the challenges related to operationalizing ethical AI principles and presents an integrated view that covers high-level ethical AI principles, the general notion of trust/trustworthiness, and product/process support in the context of responsible AI, which helps improve both trust and trustworthiness of AI for a wider se…
RWTH Aachen University（亚琛工业大学）；GESIS - Leibniz Institute for the Social Sciences（GESIS - 莱布尼茨社会科学研究所）
Measures of algorithmic fairness often do not account for human perceptions of fairness that can substantially vary between different sociodemographics and stakeholders. The FairCeptron framework is an approach for studying perceptions of fairness in algorithmic decision making such as in ranking or classification. It supports (i) studying human perceptions of fairness and (ii) comparing these human perceptions with measures of algorithmic fairness. The framework includes fairness scenario generation, fairness perception elicitation and fairness perception analysis. We demonstrate the FairCept…
Responsible AI is becoming critical as AI is widely used in our everyday lives. Many companies that deploy AI publicly state that when training a model, we not only need to improve its accuracy, but also need to guarantee that the model does not discriminate against users (fairness), is resilient to noisy or poisoned data (robustness), is explainable, and more. In addition, these objectives are not only relevant to model training, but to all steps of end-to-end machine learning, which include data collection, data cleaning and validation, model training, model evaluation, and model management …
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Northeastern University（东北大学）
Faced with the scale and surge of misinformation on social media, many platforms and fact-checking organizations have turned to algorithms for automating key parts of misinformation detection pipelines. While offering a promising solution to the challenge of scale, the ethical and societal risks associated with algorithmic misinformation detection are not well-understood. In this paper, we employ and extend upon the notion of informational justice to develop a framework for explicating issues of justice relating to representation, participation, distribution of benefits and burdens, and credib…
University of Bristol（布里斯托尔大学）；RMIT University（皇家墨尔本理工大学）
Today, artificial intelligence systems driven by machine learning algorithms can be in a position to take important, and sometimes legally binding, decisions about our everyday lives. In many cases, however, these systems and their actions are neither regulated nor certified. To help counter the potential harm that such algorithms can cause we developed an open source toolbox that can analyse selected fairness, accountability and transparency aspects of the machine learning process: data (and their features), models and predictions, allowing to automatically and objectively report them to rele…
This paper summarizes and evaluates various approaches, methods, and techniques for pursuing fairness in artificial intelligence (AI) systems. It examines the merits and shortcomings of these measures and proposes practical guidelines for defining, measuring, and preventing bias in AI. In particular, it cautions against some of the simplistic, yet common, methods for evaluating bias in AI systems, and offers more sophisticated and effective alternatives. The paper also addresses widespread controversies and confusions in the field by providing a common language among different stakeholders of …
Federated learning is an emerging privacy-preserving AI technique where clients (i.e., organisations or devices) train models locally and formulate a global model based on the local model updates without transferring local data externally. However, federated learning systems struggle to achieve trustworthiness and embody responsible AI principles. In particular, federated learning systems face accountability and fairness challenges due to multi-stakeholder involvement and heterogeneity in client data distribution. To enhance the accountability and fairness of federated learning systems, we pre…
Arizona State University（亚利桑那州立大学）；IBM Research – Thomas J. Watson Research Center（IBM研究院——托马斯·J·沃森研究中心）
In the current era, people and society have grown increasingly reliant on artificial intelligence (AI) technologies. AI has the potential to drive us towards a future in which all of humanity flourishes. It also comes with substantial risks for oppression and calamity. Discussions about whether we should (re)trust AI have repeatedly emerged in recent years and in many quarters, including industry, academia, healthcare, services, and so on. Technologists and AI researchers have a responsibility to develop trustworthy AI systems. They have responded with great effort to design more responsible A…
Fairness and Bias in Algorithmic Hiring: a Multidisciplinary Survey
Max Planck Institute for Security and Privacy（安全与隐私研究所）；Radboud University（拉德博德大学）；Eindhoven University of Technology（埃因霍温理工大学）；Randstad（Randstad公司）；European University Viadrina（弗赖堡大学）
Legislation and public sentiment throughout the world have promoted fairness metrics, explainability, and interpretability as prescriptions for the responsible development of ethical artificial intelligence systems. Despite the importance of these three pillars in the foundation of the field, they can be challenging to operationalize and attempts to solve the problems in production environments often feel Sisyphean. This difficulty stems from a number of factors: fairness metrics are computationally difficult to incorporate into training and rarely alleviate all of the harms perpetrated by the…
Heinrich Heine University Düsseldorf（海因里希·海涅杜塞尔多夫大学）
Algorithmic decision-making (ADM) increasingly shapes people's daily lives. Given that such autonomous systems can cause severe harm to individuals and social groups, fairness concerns have arisen. A human-centric approach demanded by scholars and policymakers requires taking people's fairness perceptions into account when designing and implementing ADM. We provide a comprehensive, systematic literature review synthesizing the existing empirical insights on perceptions of algorithmic fairness from 39 empirical studies spanning multiple domains and scientific disciplines. Through thorough codin…
University of Copenhagen（哥本哈根大学）；LUT University（拉乌olu大学）
The development of Artificial Intelligence (AI), including AI in Science (AIS), should be done following the principles of responsible AI. Progress in responsible AI is often quantified through evaluation metrics, yet there has been less work on assessing the robustness and reliability of the metrics themselves. We reflect on prior work that examines the robustness of fairness metrics for recommender systems as a type of AI application and summarise their key takeaways into a set of non-exhaustive guidelines for developing reliable metrics of responsible AI. Our guidelines apply to a broad spe…
There is mounting public concern over the influence that AI based systems has in our society. Coalitions in all sectors are acting worldwide to resist hamful applications of AI. From indigenous people addressing the lack of reliable data, to smart city stakeholders, to students protesting the academic relationships with sex trafficker and MIT donor Jeffery Epstein, the questionable ethics and values of those heavily investing in and profiting from AI are under global scrutiny. There are biased, wrongful, and disturbing assumptions embedded in AI algorithms that could get locked in without inte…
Shanghai Jiao Tong University（上海交通大学）；University of Michigan-Shanghai Jiao Tong University Joint Institute（上海交通大学密西根学院）
Decision support systems (e.g., for ecological conservation) and autonomous systems (e.g., adaptive controllers in smart cities) start to be deployed in real applications. Although their operations often impact many users or stakeholders, no fairness consideration is generally taken into account in their design, which could lead to completely unfair outcomes for some users or stakeholders. To tackle this issue, we advocate for the use of social welfare functions that encode fairness and present this general novel problem in the context of (deep) reinforcement learning, although it could possib…
In a world of daily emerging scientific inquisition and discovery, the prolific launch of machine learning across industries comes to little surprise for those familiar with the potential of ML. Neither so should the congruent expansion of ethics-focused research that emerged as a response to issues of bias and unfairness that stemmed from those very same applications. Fairness research, which focuses on techniques to combat algorithmic bias, is now more supported than ever before. A large portion of fairness research has gone to producing tools that machine learning practitioners can use to a…
Machine learning best practice statements have proliferated, but there is a lack of consensus on what the standards should be. For fairness standards in particular, there is little guidance on how fairness might be achieved in practice. Specifically, fairness in errors (both false negatives and false positives) can pose a problem of how to set weights, how to make unavoidable tradeoffs, and how to judge models that present different kinds of errors across racial groups. This paper considers the consequences of having higher rates of false positives for one racial group and higher rates of fals…
University of Toronto（多伦多大学）；ETH Zürich（苏黎世联邦理工学院）；Carnegie Mellon University（卡内基梅隆大学）
Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases. Our human-in-the-loop approach is designed to learn the appropriate parameters of the EOP fam…
Integrating Generative AI-enhanced Cognitive Systems in Higher Education: From Stakeholder Perceptions to a Conceptual Framework considering the EU AI Act
Faculty of Information Technology and Electrical Engineering, University of Oulu（奥卢大学信息技术与电气工程学院）
Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Schwartz Reisman Institute for Technology and Society（施瓦茨·赖斯曼技术与社会研究所）；University of Waterloo（滑铁卢大学）
