有根据的（无根据的）真相：人类-算法决策中代理标签的因果框架
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
AI总结：
本文提出一个因果框架，系统识别并厘清人类-AI决策中代理标签的五种目标变量偏差来源，用以审视既有建模假设和实验设计，并为未来研究提供评估策略。
摘要：
越来越多关于人类-AI决策的文献在研究将人类判断与统计模型相结合以改进决策的策略。该领域的研究通常通过展示在“真实标签”上预测性能的提升来评估对模型、界面或工作流程的改进。然而，这种做法忽视了人类判断与模型预测之间的一个关键区别。人类在决策中会对更广泛的现象进行推理——包括无法直接观测的潜在构念，例如疾病状态、在线评论的“毒性”或未来的“工作表现”——而预测模型则针对现有数据集中容易获得的代理标签。预测模型对简单化代理的依赖使其容易受到各种统计偏差来源的影响。在本文中，我们识别了五种目标变量偏差来源，这些偏差可能影响人类-AI决策任务中代理标签的有效性。我们开发了一个因果框架来厘清每种偏差之间的关系，并阐明在特定人类-AI决策任务中哪些偏差值得关注。我们展示了如何利用该框架来阐明先前建模工作中所做的隐含假设，并推荐用于验证这些假设在实践中是否成立的评估策略。随后，我们利用该框架重新审视了先前调查人类-AI决策的人类受试者实验设计，发现只有一小部分研究考察了与目标变量偏差相关的因素。最后，我们讨论了在未来研究中更好地应对目标变量偏差的机会。
英文摘要：
A growing literature on human-AI decision-making investigates strategies for combining human judgment with statistical models to improve decision-making. Research in this area often evaluates proposed improvements to models, interfaces, or workflows by demonstrating improved predictive performance on "ground truth" labels. However, this practice overlooks a key difference between human judgments and model predictions. Whereas humans reason about broader phenomena of interest in a decision -- including latent constructs that are not directly observable, such as disease status, the "toxicity" of online comments, or future "job performance" -- predictive models target proxy labels that are readily available in existing datasets. Predictive models' reliance on simplistic proxies makes them vulnerable to various sources of statistical bias. In this paper, we identify five sources of target variable bias that can impact the validity of proxy labels in human-AI decision-making tasks. We develop a causal framework to disentangle the relationship between each bias and clarify which are of concern in specific human-AI decision-making tasks. We demonstrate how our framework can be used to articulate implicit assumptions made in prior modeling work, and we recommend evaluation strategies for verifying whether these assumptions hold in practice. We then leverage our framework to re-examine the designs of prior human subjects experiments that investigate human-AI decision-making, finding that only a small fraction of studies examine factors related to target variable bias. We conclude by discussing opportunities to better address target variable bias in future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
