When predictions support decisions they may influence the outcome they aim to predict. We call such predictions performative; the prediction influences the target. Performativity is a well-studied phenomenon in policy-making that has so far been neglected in supervised learning. When ignored, performativity surfaces as undesirable distribution shift, routinely addressed with retraining.
We develop a risk minimization framework for performative prediction bringing together concepts from statistics, game theory, and causality. A conceptual novelty is an equilibrium notion we call performative stability. Performative stability implies that the predictions are calibrated not against past outcomes, but against the future outcomes that manifest from acting on the prediction. Our main results are necessary and sufficient conditions for the convergence of retraining to a performatively stable point of nearly minimal loss.
In full generality, performative prediction strictly subsumes the setting known as strategic classification. We thus also give the first sufficient conditions for retraining to overcome strategic feedback effects.
In performative prediction, predictions guide decision-making and hence can influence the distribution of future data. To date, work on performative prediction has focused on finding performatively stable models, which are the fixed points of repeated retraining. However, stable solutions can be far from optimal when evaluated in terms of the performative risk, the loss experienced by the decision maker when deploying a model. In this paper, we shift attention beyond performative stability and focus on optimizing the performative risk directly. We identify a natural set of properties of the lo…
Many popular algorithmic fairness measures depend on the joint distribution of predictions, outcomes, and a sensitive feature like race or gender. These measures are sensitive to distribution shift: a predictor which is trained to satisfy one of these fairness definitions may become unfair if the distribution changes. In performative prediction settings, however, predictors are precisely intended to induce distribution shift. For example, in many applications in criminal justice, healthcare, and consumer finance, the purpose of building a predictor is to reduce the rate of adverse outcomes suc…
Feedzai（Feedzai公司）；Instituto Superior Técnico, Universidade de Lisboa（里斯本大学高等技术学院）；Instituto de Telecomunicações（电信研究所）
The unparalleled ability of machine learning algorithms to learn patterns from data also enables them to incorporate biases embedded within. A biased model can then make decisions that disproportionately harm certain groups in society. Much work has been devoted to measuring unfairness in static ML environments, but not in dynamic, performative prediction ones, in which most real-world use cases operate. In the latter, the predictive model itself plays a pivotal role in shaping the distribution of the data. However, little attention has been heeded to relating unfairness to these interactions.…
Adaptive Gradient-Based Methods for a Broader Class of Optimization Problems under Performative Prediction
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）；Institute of Science Tokyo（东京科学大学）；Communication Science Laboratories, NTT, Inc.（NTT公司通信科学实验室）；Center for Advanced Intelligence Project, RIKEN（理化学研究所先进智能项目中心）
In performative prediction, the choice of a model influences the distribution of future data, typically through actions taken based on the model's predictions. We initiate the study of stochastic optimization for performative prediction. What sets this setting apart from traditional stochastic optimization is the difference between merely updating model parameters and deploying the new model. The latter triggers a shift in the distribution that affects future data, while the former keeps the distribution as is. Assuming smoothness and strong convexity, we prove rates of convergence for both gr…
Learning problems commonly exhibit an interesting feedback mechanism wherein the population data reacts to competing decision makers' actions. This paper formulates a new game theoretic framework for this phenomenon, called "multi-player performative prediction". We focus on two distinct solution concepts, namely (i) performatively stable equilibria and (ii) Nash equilibria of the game. The latter equilibria are arguably more informative, but can be found efficiently only when the game is monotone. We show that under mild assumptions, the performatively stable equilibria can be found efficient…
Performative Prediction with Bandit Feedback: Learning through Reparameterization
University of California, Santa Cruz（加州大学圣克鲁兹分校）；Columbia University（哥伦比亚大学）；the Chinese University of Hong Kong（香港中文大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
Deployed supervised machine learning models make predictions that interact with and influence the world. This phenomenon is called performative prediction by Perdomo et al. (ICML 2020). It is an ongoing challenge to understand the influence of such predictions as well as design tools so as to control that influence. We propose a theoretical framework where the response of a target population to the deployed classifier is modeled as a function of the classifier and the current state (distribution) of the population. We show necessary and sufficient conditions for convergence to an equilibrium o…
The recent framework of performative prediction is aimed at capturing settings where predictions influence the target/outcome they want to predict. In this paper, we introduce a natural multi-agent version of this framework, where multiple decision makers try to predict the same outcome. We showcase that such competition can result in interesting phenomena by proving the possibility of phase transitions from stability to instability and eventually chaos. Specifically, we present settings of multi-agent performative prediction where under sufficient conditions their dynamics lead to global stab…
Decision-makers often act in response to data-driven predictions, with the goal of achieving favorable outcomes. In such settings, predictions don't passively forecast the future; instead, predictions actively shape the distribution of outcomes they are meant to predict. This performative prediction setting raises new challenges for learning "optimal" decision rules. In particular, existing solution concepts do not address the apparent tension between the goals of forecasting outcomes accurately and steering individuals to achieve desirable outcomes. To contend with this concern, we introduce …
We construct a model of expert prediction where predictions can influence the state of the world. Under this model, we show through theoretical and numerical results that proper scoring rules can incentivize experts to manipulate the world with their predictions. We also construct a simple class of scoring rules that avoids this problem.
Performative Risk Control: Calibrating Models for Reliable Deployment under Performativity
New York University（纽约大学）；University of California, Los Angeles（加州大学洛杉矶分校）；Simon Fraser University（西蒙弗雷泽大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Robust and Reliable AI for Predictive Quality in Semiconductor Materials Manufacturing with MLOps and Uncertainty Quantification
Merck Group（默克集团）；Versum Materials US LLC（Versum Materials美国分公司）；Merck Electronics KGaA（默克电子（KGaA））；Technical University of Darmstadt（达姆施塔特技术大学）；Otto von Guericke University Magdeburg（奥托·冯·格里克大学马格德堡分校）
The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this work, we propose a geometric gradient analysis (GGA) to improve the identification of untrustworthy predictions without retraining of a given model. GGA analyzes the geometry of the loss landscape of…
AICO: Feature Significance Tests for Supervised Learning
Stanford University, Department of Management Science and Engineering and Institute for Computational and Mathematical Engineering（斯坦福大学管理科学与工程系和计算与数学工程研究所）；Upstart, Inc.（Upstart公司）；Stanford University, Institute for Computational and Mathematical Engineering（斯坦福大学计算与数学工程研究所）
We propose a novel prediction interval (PI) method for uncertainty quantification, which addresses three major issues with the state-of-the-art PI methods. First, existing PI methods require retraining of neural networks (NNs) for every given confidence level and suffer from the crossing issue in calculating multiple PIs. Second, they usually rely on customized loss functions with extra sensitive hyperparameters for which fine tuning is required to achieve a well-calibrated PI. Third, they usually underestimate uncertainties of out-of-distribution (OOD) samples leading to over-confident PIs. O…
Max-Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；University of California, Berkeley（加州大学伯克利分校）；University of Michigan（密歇根大学）
Predictions about people, such as their expected educational achievement or their credit risk, can be performative and shape the outcome that they aim to predict. Understanding the causal effect of these predictions on the eventual outcomes is crucial for foreseeing the implications of future predictive models and selecting which models to deploy. However, this causal estimation task poses unique challenges: model predictions are usually deterministic functions of input features and highly correlated with outcomes. This can make the causal effects of predictions on outcomes impossible to disen…
A Dynamic Model of Performative Human-ML Collaboration: Theory and Empirical Evidence
MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）；Tübingen AI Center（蒂宾根人工智能中心）；Harvard Business School（哈佛商学院）；Harvard University（哈佛大学）；CEPR（欧洲经济政策研究中心）；NBER（美国国家经济研究局）
