Machine learning systems have become popular in fields such as marketing, financing, or data mining. While they are highly accurate, complex machine learning systems pose challenges for engineers and users. Their inherent complexity makes it impossible to easily judge their fairness and the correctness of statistically learned relations between variables and classes. Explainable AI aims to solve this challenge by modelling explanations alongside with the classifiers, potentially improving user trust and acceptance. However, users should not be fooled by persuasive, yet untruthful explanations. We therefore conduct a user study in which we investigate the effects of model accuracy and explanation fidelity, i.e. how truthfully the explanation represents the underlying model, on user trust. Our findings show that accuracy is more important for user trust than explainability. Adding an explanation for a classification result can potentially harm trust, e.g. when adding nonsensical explanations. We also found that users cannot be tricked by high-fidelity explanations into having trust for a bad classifier. Furthermore, we found a mismatch between observed (implicit) and self-reported (explicit) trust.
Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations
Research Center Trustworthy Data Science and Security(可信数据科学与安全研究所以);TU Dortmund University(图卢兹大学);University Duisburg-Essen(杜伊斯堡- Essen大学);University of Nottingham(诺丁汉大学)
Measure Utility, Gain Trust: Practical Advice for XAI Researcher
School of Engineering and Applied Sciences Washington State University(华盛顿州立大学工程与应用科学学院);Data Sciences and Analytics Group Pacific Northwest National Laboratory(太平洋西北国家实验室数据科学与分析组);Cognitive Engineering Center Georgia Institute of Technology(佐治亚理工学院认知工程中心);Visual Analytics Group Pacific Northwest National Laboratory(太平洋西北国家实验室可视化分析组)
CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models
UCLA(加州大学洛杉矶分校);University of Michigan(密歇根大学);Oregon State University(俄勒冈州立大学);Beijing Institute for General AI (BIGAI)(北京通用人工智能研究院);Tsinghua University(清华大学);Peking University(北京大学)
The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
Context-dependent Explainability and Contestability for Trustworthy Medical Artificial Intelligence: Misclassification Identification of Morbidity Recognition Models in Preterm Infants
Scientific Explanations in Health Sciences: Causality, Trust, and Epistemic Adequacy
College of Mathematical Medicine, Zhejiang Normal University(浙江师范大学数学医学院);DAIS, Ca’ Foscari University of Venice(威尼斯大学DAIS);European Centre for Living Technology(欧洲生命技术中心)
Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review
School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Institute of Automotive Engineering, Technical University (TU) of Darmstadt(达姆施塔特工业大学汽车工程研究所);School of Engineering and Physical Sciences, Heriot-Watt University(赫瑞瓦特大学工程与物理科学学院)
To Explain or Not to Explain: A Study on the Necessity of Explanations for Autonomous Vehicles
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Stanford University(斯坦福大学);Apple Inc.(苹果公司)
在自动驾驶汽车等自动系统的背景下,可解释的人工智能引起了研究人员的广泛兴趣。最近的研究发现,为自动驾驶汽车的行为提供解释有很多好处(例如,增加信任和接受度),但很少强调何时需要解释以及解释的内容如何随驾驶上下文而变化。在这项工作中,我们调查的情况下,人们需要解释,以及如何解释的关键程度与情况和驱动程序类型的变化。通过用户实验,我们要求参与者评估解释的必要性,并测量在不同背景下对他们对自动驾驶汽车信任的影响。此外,我们提出了一个自动驾驶解释数据集,其中包含第一人称解释和1103个视频片段的必要性的相关度量,增强了Berkeley Deep Drive Attention数据集。我们的研究表明,驾驶员类型和驾驶场景决定了是否需要解释。特别是,人们倾向于同意的必要性,近碰撞事件,但持有不同的意见,正常或异常的驾驶情况。