Various forms of implications of artificial intelligence that either exacerbate or decrease racial systemic injustice have been explored in this applied research endeavor. Taking each thematic area of identifying, analyzing, and debating an systemic issue have been leveraged in investigating merits and drawbacks of using algorithms to automate human decision making in racially sensitive environments. It has been asserted through the analysis of historical systemic patterns, implicit biases, existing algorithmic risks, and legal implications that natural language processing based AI, such as risk assessment tools, have racially disparate outcomes. It is concluded that more litigative policies are needed to regulate and restrict how internal government institutions and corporations utilize algorithms, privacy and security risks, and auditing requirements in order to diverge from racially injustice outcomes and practices of the past.
Early studies of risk assessment algorithms used in criminal justice revealed widespread racial biases. In response, machine learning researchers have developed methods for fairness, many of which rely on equalizing empirical metrics across protected attributes. Here, I recall sociotechnical perspectives to delineate the significant gap between fairness in theory and practice, focusing on criminal justice. I (1) illustrate how social context can undermine analyses that are restricted to an AI system's outputs, and (2) argue that much of the fair ML literature fails to account for epistemologic…
This two-part paper argues that seemingly "technical" choices made by developers of machine-learning based algorithmic tools used to inform decisions by criminal justice authorities can create serious constitutional dangers, enhancing the likelihood of abuse of decision-making power and the scope and magnitude of injustice. Drawing on three algorithmic tools in use, or recently used, to assess the "risk" posed by individuals to inform how they should be treated by criminal justice authorities, we integrate insights from data science and public law scholarship to show how public law principles …
This two part paper argues that seemingly "technical" choices made by developers of machine-learning based algorithmic tools used to inform decisions by criminal justice authorities can create serious constitutional dangers, enhancing the likelihood of abuse of decision-making power and the scope and magnitude of injustice. Drawing on three algorithmic tools in use, or recently used, to assess the "risk" posed by individuals to inform how they should be treated by criminal justice authorities, we integrate insights from data science and public law scholarship to show how public law principles …
Open data promotes transparency and accountability as everyone can analyse it. Law enforcement and the judiciary are increasingly making data available, to increase trust and confidence in the criminal justice system. Due to privacy legislation, judicial open data -- like court judgments -- in Europe is usually anonymised. Because this removes part of the information on for instance offenders, the question arises to what extent criminological research into sentencing can make use of anonymised open data. We answer this question based on a case study in which we use the open data of the Dutch c…
Big data and algorithmic risk prediction tools promise to improve criminal justice systems by reducing human biases and inconsistencies in decision making. Yet different, equally-justifiable choices when developing, testing, and deploying these sociotechnical tools can lead to disparate predicted risk scores for the same individual. Synthesizing diverse perspectives from machine learning, statistics, sociology, criminology, law, philosophy and economics, we conceptualize this phenomenon as predictive inconsistency. We describe sources of predictive inconsistency at different stages of algorith…
We present a survey of tools used in the criminal justice system in the UK in three categories: data infrastructure, data analysis, and risk prediction. Many tools are currently in deployment, offering potential benefits, including improved efficiency and consistency. However, there are also important concerns. Transparent information about these tools, their purpose, how they are used, and by whom is difficult to obtain. Even when information is available, it is often insufficient to enable a satisfactory evaluation. More work is needed to establish governance mechanisms to ensure that tools …
Much attention has been paid to algorithms related to sentencing, the setting of bail, parole decisions and recidivism while less attention has been paid to carceral algorithms, those algorithms used to determine an incarcerated individual's lived experience. In this paper we study one such algorithm, the Pennsylvania Additive Classification Tool (PACT) that assigns custody levels to incarcerated individuals. We analyze the PACT in ways that criminal justice algorithms are often analyzed: namely, we train an accurate machine learning model for the PACT; we study its fairness across sex, age an…
In the criminal justice system, algorithmic risk assessment instruments are used to predict the risk a defendant poses to society; examples include the risk of recidivating or the risk of failing to appear at future court dates. However, defendants are also at risk of harm from the criminal justice system. To date, there exists no risk assessment instrument that considers the risk the system poses to the individual. We develop a risk assessment instrument that "flips the script." Using data about U.S. federal sentencing decisions, we build a risk assessment instrument that predicts the likelih…
Machine learning best practice statements have proliferated, but there is a lack of consensus on what the standards should be. For fairness standards in particular, there is little guidance on how fairness might be achieved in practice. Specifically, fairness in errors (both false negatives and false positives) can pose a problem of how to set weights, how to make unavoidable tradeoffs, and how to judge models that present different kinds of errors across racial groups. This paper considers the consequences of having higher rates of false positives for one racial group and higher rates of fals…
The Imperishable Record, Forgiveness, and the Future of Criminal Justice
Graduate School of Law, Kyushu University（九州大学法学研究科）；Vinted（Vinted公司）；Vilnius University Law Faculty（维尔纽斯大学法学院）；Center for Advanced Studies in Bioscience Innovation Law (CeBIL) at the University of Copenhagen（哥本哈根大学生物科学创新法高级研究中心）；National University of Singapore’s Centre for Biomedical Ethics（新加坡国立大学生物医学伦理中心）
I discuss here three important roles where machine intelligence, brain and behaviour studies together may facilitate criminal law. First, predictive modelling using brain and behaviour data may support legal investigations by predicting categorical, continuous, and longitudinal legal outcomes of interests related to brain injury and mental illnesses. Second, psychological, psychiatric, and behavioural studies supported by machine learning algorithms may help predict human behaviour and actions, such as lies, biases, and visits to crime scenes. Third, machine learning models have been used to p…
Risk assessment algorithms in criminal justice put people's lives at the discretion of a simple statistical tool. This thesis explores how algorithmic decision-making in criminal policy can exhibit feedback effects, where disadvantage accumulates among those deemed 'high risk' by the state. Evidence from Philadelphia suggests that risk - and, by extension, criminality - is not fundamental or in any way exogenous to political decision-making. A close look at the geographical and demographic properties of risk calls into question the current practice of prediction in criminal policy. Using court…
Breaking the Cycle of Incarceration With Targeted Mental Health Outreach: A Case Study in Machine Learning for Public Policy
Machine Learning Department（机器学习部门）；Heinz College of Public Policy（公共政策学院）；Carnegie Mellon University（卡内基梅隆大学）；Center for Data Science and Public Policy（数据科学与公共政策中心）；University of Chicago（芝加哥大学）
Algorithmic audits have been embraced as tools to investigate the functioning and consequences of sociotechnical systems. Though the term is used somewhat loosely in the algorithmic context and encompasses a variety of methods, it maintains a close connection to audit studies in the social sciences--which have, for decades, used experimental methods to measure the prevalence of discrimination across domains like housing and employment. In the social sciences, audit studies originated in a strong tradition of social justice and participatory action, often involving collaboration between researc…
Risk assessment tools are widely used around the country to inform decision making within the criminal justice system. Recently, considerable attention has been devoted to the question of whether such tools may suffer from racial bias. In this type of assessment, a fundamental issue is that the training and evaluation of the model is based on a variable (arrest) that may represent a noisy version of an unobserved outcome of more central interest (offense). We propose a sensitivity analysis framework for assessing how assumptions on the noise across groups affect the predictive bias properties …
Currently, there is uncertainty surrounding the merits of open-source versus proprietary algorithm development. Though justification in favor of each exists, we argue that open-source algorithm development should be the standard in highly consequential contexts that affect people's lives for reasons of transparency and collaboration, which contribute to greater predictive accuracy and enjoy the additional advantage of cost-effectiveness. To make this case, we focus on criminal sentencing algorithms, as criminal sentencing is highly consequential, and impacts society and individual people. Furt…
MIT Media Lab（麻省理工学院媒体实验室）；Harvard University（哈佛大学）
Actuarial risk assessments might be unduly perceived as a neutral way to counteract implicit bias and increase the fairness of decisions made at almost every juncture of the criminal justice system, from pretrial release to sentencing, parole and probation. In recent times these assessments have come under increased scrutiny, as critics claim that the statistical techniques underlying them might reproduce existing patterns of discrimination and historical biases that are reflected in the data. Much of this debate is centered around competing notions of fairness and predictive accuracy, resting…
Carnegie Mellon University（卡内基梅隆大学）；Ctr for Data Science & Public Policy , University of Chicago（芝加哥大学数据科学与公共政策中心）；Los Angeles City Attorney’s Office（洛杉矶市检察官办公室）；Heinz College of Information Systems & Public Policy , Carnegie Mellon University（卡内基梅隆大学海因茨信息系统与公共政策学院）；Machine Learning Department , Carnegie Mellon University（卡内基梅隆大学机器学习系）；Director, Recidivism Reduction and Drug Diversion Unit (R2D2) , Los Angeles City Attorney’s Office（洛杉矶市检察官办公室再犯率降低和药物分流部门主任（R2D2））
The criminal justice system is currently ill-equipped to improve outcomes of individuals who cycle in and out of the system with a series of misdemeanor offenses. Often due to constraints of caseload and poor record linkage, prior interactions with an individual may not be considered when an individual comes back into the system, let alone in a proactive manner through the application of diversion programs. The Los Angeles City Attorney's Office recently created a new Recidivism Reduction and Drug Diversion unit (R2D2) tasked with reducing recidivism in this population. Here we describe a coll…
In our current society, secret algorithms make important decisions about individuals. There has been substantial discussion about whether these algorithms are unfair to groups of individuals. While noble, this pursuit is complex and ultimately stagnating because there is no clear definition of fairness and competing definitions are largely incompatible. We argue that the focus on the question of fairness is misplaced, as these algorithms fail to meet a more important and yet readily obtainable goal: transparency. As a result, creators of secret algorithms can provide incomplete or misleading d…
This paper will discuss the need for national-level organizational strategies to effectively combat cyber security threats and cybercrime. In many countries, new agencies have been established and/or new roles have been allotted to existing agencies to cope with the needs for cyber security or fighting against cybercrime. The two pillars of organizational structure and functions (i.e., security vs. law enforcement) have given new challenges to us, especially in the context of traditional criminal justice system. To illustrate the challenges, a case study examining the responses to major securi…
Predictive algorithms are now used to help distribute a large share of our society's resources and sanctions, such as healthcare, loans, criminal detentions, and tax audits. Under the right circumstances, these algorithms can improve the efficiency and equity of decision-making. At the same time, there is a danger that the algorithms themselves could entrench and exacerbate disparities, particularly along racial, ethnic, and gender lines. To help ensure their fairness, many researchers suggest that algorithms be subject to at least one of three constraints: (1) no use of legally protected feat…
This chapter argues for a structural injustice approach to the governance of AI. Structural injustice has an analytical and an evaluative component. The analytical component consists of structural explanations that are well-known in the social sciences. The evaluative component is a theory of justice. Structural injustice is a powerful conceptual tool that allows researchers and practitioners to identify, articulate, and perhaps even anticipate, AI biases. The chapter begins with an example of racial bias in AI that arises from structural injustice. The chapter then presents the concept of str…
Laypeople's Attitudes Towards Fair, Affirmative, and Discriminatory Decision-Making Algorithms
Max Planck Institute for Security and Privacy（马克斯·普朗克安全与隐私研究所）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；Max Planck Institute for Research on Collective Goods（马克斯·普朗克集体利益研究研究所）；University of Freiburg（弗赖堡大学）
Researchers and journalists have repeatedly shown that algorithms commonly used in domains such as credit, employment, healthcare, or criminal justice can have discriminatory effects. Some organizations have tried to mitigate these effects by simply removing sensitive features from an algorithm's inputs. In this paper, we explore the limits of this approach using a unique opportunity. In 2019, Facebook agreed to settle a lawsuit by removing certain sensitive features from inputs of an algorithm that identifies users similar to those provided by an advertiser for ad targeting, making both the m…
Our machines, products, utilities, and environments have long been monitored by embedded software systems. Our professional, commercial, social and personal lives are also subject to monitoring as they are mediated by software systems. Data on nearly everything now exists, waiting to be collected and analysed for all sorts of reasons. Given the rising tide of data we pose the questions: What is monitoring? Do diverse and disparate monitoring systems have anything in common? We attempt answer these questions by proposing an abstract conceptual framework for studying monitoring. We argue that it…
Artificial intelligence (AI) is a technology which is increasingly being utilised in society and the economy worldwide, and its implementation is planned to become more prevalent in coming years. AI is increasingly being embedded in our lives, supplementing our pervasive use of digital technologies. But this is being accompanied by disquiet over problematic and dangerous implementations of AI, or indeed, even AI itself deciding to do dangerous and problematic actions, especially in fields such as the military, medicine and criminal justice. These developments have led to concerns about whether…
Algorithmic systems have been used to inform consequential decisions for at least a century. Recidivism prediction dates back to the 1920s. Automated credit scoring dates began in the middle of the last century, but the last decade has witnessed an acceleration in the adoption of prediction algorithms. They are deployed to screen job applicants for the recommendation of products, people, and content, as well as in medicine (diagnostics and decision aids), criminal justice, facial recognition, lending and insurance, and the allocation of public services. The prominence of algorithmic methods ha…
University of Toronto（多伦多大学）；Australian National University（澳大利亚国立大学）
The societal and ethical implications of the use of opaque artificial intelligence systems for consequential decisions, such as welfare allocation and criminal justice, have generated a lively debate among multiple stakeholder groups, including computer scientists, ethicists, social scientists, policy makers, and end users. However, the lack of a common language or a multi-dimensional framework to appropriately bridge the technical, epistemic, and normative aspects of this debate prevents the discussion from being as productive as it could be. Drawing on the philosophical literature on the nat…
