人类对齐影响AI辅助决策的效用
Human-Alignment Influences the Utility of AI-assisted Decision Making
AI总结：
本研究通过大规模人类实验（n=703）证明，AI置信度与决策者自身置信度的对齐程度正向影响AI辅助决策效用，且多校准后处理可同时提升对齐度与效用。
摘要：
每当在AI辅助决策中使用AI模型预测相关的（二元）结果时，人们普遍认为，模型应随每个预测一起提供AI置信度值。然而，目前尚不清楚为什么决策者在使用AI置信度值时经常难以对何时信任预测形成良好判断。最近，Corvelo Benz和Gomez Rodriguez提出，对于理性决策者而言，AI辅助决策的效用本质上受限于AI置信度值与决策者对其自身预测的置信度之间的对齐程度。在本研究中，我们从实证角度探究对齐程度实际上在多大程度上影响AI辅助决策的效用。为此，我们设计并开展了一项大规模人类受试者研究（n=703），参与者在AI模型的辅助下解决一个简单的决策任务——在线纸牌游戏，该AI模型具有可调节的对齐程度。我们的结果显示，对齐程度与AI辅助决策的效用之间存在正相关关系。此外，我们的结果还表明，对AI置信度值进行后处理以实现相对于参与者对其自身预测置信度的多校准，既提高了对齐程度，也提升了AI辅助决策的效用。
英文摘要：
Whenever an AI model is used to predict a relevant (binary) outcome in AI-assisted decision making, it is widely agreed that, together with each prediction, the model should provide an AI confidence value. However, it has been unclear why decision makers have often difficulties to develop a good sense on when to trust a prediction using AI confidence values. Very recently, Corvelo Benz and Gomez Rodriguez have argued that, for rational decision makers, the utility of AI-assisted decision making is inherently bounded by the degree of alignment between the AI confidence values and the decision maker's confidence on their own predictions. In this work, we empirically investigate to what extent the degree of alignment actually influences the utility of AI-assisted decision making. To this end, we design and run a large-scale human subject study (n=703) where participants solve a simple decision making task - an online card game - assisted by an AI model with a steerable degree of alignment. Our results show a positive association between the degree of alignment and the utility of AI-assisted decision making. In addition, our results also show that post-processing the AI confidence values to achieve multicalibration with respect to the participants' confidence on their own predictions increases both the degree of alignment and the utility of AI-assisted decision making.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
