The field of eXplainable Artificial Intelligence (XAI) is increasingly recognizing the need to personalize and/or interactively adapt the explanation to better reflect users' explanation needs. While dialogue-based approaches to XAI have been proposed recently, the state-of-the-art in XAI is still characterized by what we call one-shot, non-personalized and one-way explanations. In contrast, dialogue-based systems that can adapt explanations through interaction with a user promise to be superior to GUI-based or dashboard explanations as they offer a more intuitive way of requesting information. In general, while interactive XAI systems are often evaluated in terms of user satisfaction, there are limited studies that access user's objective model understanding. This is in particular the case for dialogue-based XAI approaches. In this paper, we close this gap by carrying out controlled experiments within a dialogue framework in which we measure understanding of users in three phases by asking them to simulate the predictions of the model they are learning about. By this, we can quantify the level of (improved) understanding w.r.t. how the model works, comparing the state prior, and after the interaction. We further analyze the data to reveal patterns of how the interaction between groups with high vs. low understanding gain differ. Overall, our work thus contributes to our understanding about the effectiveness of XAI approaches.
Measuring What Matters: A Unified Evaluation Framework for GNN Explainability
Sapienza University of Rome（罗马萨皮恩扎大学）；Intesa Sanpaolo AI Research（Intesa Sanpaolo人工智能研究）；Anti Financial Crime Digital Hub（反金融犯罪数字中心）；Intesa Sanpaolo Innovation Center（Intesa Sanpaolo创新中心）
Measuring Perceived Trust in XAI-Assisted Decision-Making by Eliciting a Mental Model
Eindhoven University of Technology（埃因霍温理工大学）；Eindhoven Artificial Intelligence Systems Institute（埃因霍温人工智能系统研究所）；Ca’ Foscari University of Venice（威尼斯大学）
The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus
Technical University of Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；UiT Arctic University of Norway（挪威北极大学）；Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；University of Potsdam（波茨坦大学）；Leibniz Institute of Agricultural Engineering and Bioeconomy e.V.（莱布尼茨农业工程与生物经济研究所）
Explanation-by-Example Based on Item Response Theory
Universidade Federal do Pará（帕拉联邦大学）；Instituto Tecnológico Vale（淡水河谷技术研究所）；Instituto Federal do Pará（帕拉联邦理工学院）；Universidade Federal de Pernambuco（伯南布哥联邦大学）；Université Montpellier（蒙彼利埃大学）
