算法不确定性是否会影响人类专家?来自选择性大学录取的实地实验证据
Does Algorithmic Uncertainty Sway Human Experts? Evidence from a Field Experiment in Selective College Admissions
AI总结:
本文通过实地实验研究算法不确定性对人类决策的影响,发现即使算法预测存在差异,录取结果未显著变化,表明专家决策受专业判断和制度环境影响较大。
摘要:
算法预测本质上具有不确定性:即使模型整体准确率相似,对同一个体的预测可能不同,这引发了高风险决策可能受任意模型选择影响的担忧。本文定义算法敏感性为任意模型选择如何影响人类决策的程度:当决策者面对同一个体的不同算法预测时,决策结果会有多大变化。我们在美国选择性大学录取周期中进行了一项随机实地实验(n=19,545),其中招生官在审查每份申请时均需评估算法分数,同时随机改变分数来源的模型。尽管两个模型整体表现相似,但它们对同一申请人的评分经常不同,从而产生了外生的评分变化。令人惊讶的是,我们发现算法敏感性证据不足:呈现更有利的评分并未显著增加申请人的录取概率,即使模型意见存在显著分歧。这些发现表明,在这种专家、高风险的环境中,人类决策在很大程度上对任意的算法预测变化不敏感,强调了专业判断和制度环境在缓解算法不确定性的下游影响中的作用。
英文摘要:
Algorithmic predictions are inherently uncertain: even models with similar aggregate accuracy can produce different predictions for the same individual, raising concerns that high-stakes decisions may become sensitive to arbitrary modeling choices. In this paper, we define \emph{algorithmic sensitivity} as the extent to which arbitrary modeling choices propagate into human decisions: how much a decision outcome shifts when a more favorable versus less favorable algorithmic prediction is presented to the decision-maker for the same individual. We estimate this in a randomized field experiment ($n=19{,}545$) embedded in a selective U.S. college admissions cycle, in which admissions officers reviewed each application alongside an algorithmic score while we randomly varied whether the score came from one of two similarly accurate prediction models. Although the two models performed similarly in aggregate, they frequently assigned different scores to the same applicant, creating exogenous variation in the score shown. Surprisingly, we find little evidence of algorithmic sensitivity: presenting a more favorable score does not meaningfully increase an applicant's probability of admission on average, even when the models disagree substantially. These findings suggest that, in this expert, high-stakes setting, human decision-making is largely invariant to arbitrary variation in algorithmic predictions, underscoring the role of professional discretion and institutional context in mediating the downstream effects of algorithmic uncertainty.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。