Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model's to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately \textit{calibrate} human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI's errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI.
Exploring Trust Calibration in XAI - The Impact of Exposing Model Limitations to Lay Users
Research Center Trustworthy Data Science and Security（可信数据科学与安全研究中心）；University Alliance Ruhr（鲁尔大学联盟）；University of Duisburg-Essen（杜伊斯堡- Essen大学）；TU Dortmund University（多特蒙德技术大学）
An Experimental Comparison of Cognitive Forcing Functions for Execution Plans in AI-Assisted Writing: Effects On Trust, Overreliance, and Perceived Critical Thinking
Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；Microsoft Research（微软研究院）
Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model
Eindhoven University of Technology（埃因霍温理工大学）；Eindhoven Artificial Intelligence Systems Institute（埃因霍温人工智能系统研究所）；Ca’ Foscari University of Venice（威尼斯大学）
Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations
Research Center Trustworthy Data Science and Security（可信数据科学与安全研究所以）；TU Dortmund University（图卢兹大学）；University Duisburg-Essen（杜伊斯堡- Essen大学）；University of Nottingham（诺丁汉大学）
Facilitating automated conversion of scientific knowledge into scientific simulation models with the Machine Assisted Generation, Calibration, and Comparison (MAGCC) Framework
University of Vermont Larner College of Medicine（佛蒙特大学拉纳医学院）
Physical Color Calibration of Digital Pathology Scanners for Robust Artificial Intelligence Assisted Cancer Diagnosis
Karolinska Institutet（卡罗林斯卡学院）；FFEI Ltd（FFEI有限公司）；University of Turku（图尔库大学）；Stavanger University Hospital（斯塔万格大学医院）；University of Stavanger（斯塔万格大学）；Aarhus University Hospital（奥胡斯大学医院）；Aarhus University（奥胡斯大学）；Tampere University（坦佩雷大学）
To Trust or Not to Trust: On Calibration in ML-based Resource Allocation for Wireless Networks
Centre for Wireless Innovation, Queen’s University of Belfast（女王大学贝尔法斯特无线创新中心）；Dhali Holdings Ltd.（Dhali控股有限公司）；School of Electrical and Computer Engineering, University of Campinas（坎皮纳斯大学电气与计算机工程学院）；Faculty of Engineering and Applied Science, Memorial University（纪念大学工程与应用科学学院）；School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast（女王大学贝尔法斯特电子、电气工程与计算机科学学院）
TRUST-ESD: A Risk-Calibrated and Governance-Aware AI Framework for Enterprise Strategic Decision Support Under Uncertainty
School of Economics and Management, Xinyu University（新余学院经济与管理学院）；College of Computer Science, Chongqing University（重庆大学计算机科学学院）；School of Economics and Business Administration, Chongqing University（重庆大学经济与工商管理学院）；School of Management, Xi’an Jiaotong University（西安交通大学管理学院）；School of Law and Public Administration, China Three Gorges University（三峡大学法学与公共管理学院）
