AI 中文总结
本研究针对算法辅助人类决策中重复接触算法的行为适应问题，定义了适配的效应估计量，提出极小极大阶梯双楔设计，验证了其优于两种常见替代设计。
AI 中文摘要
在医疗等高风险场景的算法辅助决策中，算法决策支持工具提供建议，但最终决策由人类做出。部署前确定算法辅助是否真正提升人类决策质量至关重要，随机实验是收集可靠证据的方式之一。然而，过往实验设计与分析忽略了决策行为会随重复接触算法而适应。本研究定义了一组效应估计量，用于刻画重复接触下的人类行为适应，并论证这些估计量对理解算法辅助影响的作用。我们提出了极小极大阶梯双楔设计，以利于估计目标效应估计量。最后，我们将该设计与通过回顾过往算法辅助随机试验确定的两种常见替代设计进行比较，结果显示，在三种受现实动态启发的行为适应形式——自动化偏差、警觉疲劳与校准依赖下，这两种替代设计更难估计目标效应估计量，且会产生有偏估计。
英文摘要
In algorithm-assisted decision-making in high-stakes settings like healthcare, an algorithmic decision support tool provides a recommendation, but the human ultimately makes the decision. Determining whether algorithm assistance actually improves the quality of human decision-making prior to deployment is critical, and randomized experiments are one way to collect robust evidence. Historically, however, experimental designs and analyses ignore how decision-making behavior adapts with repeated algorithm exposure. In this work, we define a set of effect estimands that account for and characterize human behavior adaptation under repeated exposure and justify why these estimands are useful to target for developing a better understanding of the impact of algorithm assistance. We propose a minimax stepped double wedge design that facilitates estimating these target estimands. Finally, we compare our proposed design to two common alternative designs identified through a review of historical randomized trials of algorithm assistance. We show that these designs are less amenable to estimating the target estimands and produce biased estimates under three different forms of behavioral adaptation inspired by dynamics observed in the real world -- automation bias, alert fatigue, and calibrated reliance.
