Benchmarking XAI Explanations with Human-Aligned Evaluations
Rémi Kazmierczak, Steve Azzolin, Eloïse Berthier, Anna Hedström, Patricia Delhomme, David Filliat, Nicolas Bousquet, Goran Frehse, Massimiliano Mancini, Baptist… 展开作者
Rémi Kazmierczak, Steve Azzolin, Eloïse Berthier, Anna Hedström, Patricia Delhomme, David Filliat, Nicolas Bousquet, Goran Frehse, Massimiliano Mancini, Baptiste Caramiaux, Andrea Passerini, Gianni Franchi
We introduce PASTA (Perceptual Assessment System for explanaTion of Artificial Intelligence), a novel human-centric framework for evaluating eXplainable AI (XAI) techniques in computer vision. Our first contribution is the creation of the PASTA-dataset, the first large-scale benchmark that spans a diverse set of models and both saliency-based and concept-based explanation methods. This dataset enables robust, comparative analysis of XAI techniques based on human judgment. Our second contribution is an automated, data-driven benchmark that predicts human preferences using the PASTA-dataset. This scoring called PASTA-score method offers scalable, reliable, and consistent evaluation aligned with human perception. Additionally, our benchmark allows for comparisons between explanations across different modalities, an aspect previously unaddressed. We then propose to apply our scoring method to probe the interpretability of existing models and to build more human interpretable XAI methods.
Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
Human Technology Institute, University of Technology Sydney（悉尼科技大学人类技术研究所）；School of Information Systems, Queensland University of Technology（昆士兰科技大学信息系统学院）；INESC-ID Lisboa / Instituto Superior Técnico, ULisboa（INESC-ID里斯本/里斯本大学高等技术学院）
Benchmarking Class Activation Map Methods for Explainable Brain Hemorrhage Classification on Hemorica Dataset
Faculty of Computer Engineering, K. N. Toosi University of Technology（计算机工程学院，K.N.托菲大学）；Faculty of Electrical Engineering, K. N. Toosi University of Technology（电气工程学院，K.N.托菲大学）；Faculty of Mechanical Engineering, Tarbiat Modares University（机械工程学院，塔里比亚特莫达res大学）
When Bias Backfires: The Modulatory Role of Counterfactual Explanations on the Adoption of Algorithmic Bias in XAI-Supported Human Decision-Making
Machine Learning Group, Bielefeld University（比勒菲尔德大学机器学习组）；Research Center Trustworthy Data Science and Security of the University Alliance Ruhr, Faculty of Informatics, Technical University Dortmund（鲁尔大学联盟可信数据科学与安全研究所以及多特蒙德技术大学信息学院）
Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI
Aristotle University of Thessaloniki, School of Informatics（塞萨洛尼基亚里士多德大学信息学院）；Technical University of Denmark（丹麦技术大学）；Telefonica Scientific Research（电信科学研究院）
Scientific Explanations in Health Sciences: Causality, Trust, and Epistemic Adequacy
College of Mathematical Medicine, Zhejiang Normal University（浙江师范大学数学医学院）；DAIS, Ca’ Foscari University of Venice（威尼斯大学DAIS）；European Centre for Living Technology（欧洲生命技术中心）
FG-CXR: A Radiologist-Aligned Gaze Dataset for Enhancing Interpretability in Chest X-Ray Report Generation
University of Arkansas（阿肯色大学）；West Virginia University（西弗吉尼亚大学）；University of Liverpool（利物浦大学）；MD Anderson Cancer Center（MD安德森癌症中心）；University of Houston（休斯顿大学）
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
State Key Lab of Software Development Environment, Beihang University（北京航空航天大学软件开发环境国家重点实验室）；SenseTime（商汤科技）；Berkeley Artificial Intelligence Research Lab, University of California, Berkeley（加州大学伯克利分校伯克利人工智能研究实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Oxford（牛津大学）；JD Explore Academy（京东探索研究院）
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Technion – Israel Institute of Technology（技术ion – 以色列理工学院）；Prior Labs（Prior实验室）；NVIDIA；SODA Team, INRIA Saclay, Palaiseau（SODA团队，INRIA萨克莱，帕莱索）；University of Freiburg（弗赖堡大学）；Probabl；ELLIS Institute Tübingen（图宾根ELLIS研究所）
Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks
Hong Kong University of Science and Technology（香港科技大学）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；Harvard University（哈佛大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Technical University Dresden（德累斯顿工业大学）；Medical University of Vienna（维也纳医科大学）；Pontificia Universidad Católica de Chile（智利天主教大学）；Hospital Clínic of Barcelona（巴塞罗那临床医院）；University of Barcelona（巴塞罗那大学）；National & Kapodistrian University of Athens（雅典国立卡波季斯特里安大学）；Sestre milosrdnice University Hospital Center（米洛塞尔德尼采大学医院中心）；Derma Style（Derma Style皮肤科诊所）；Dubai London Clinic（迪拜伦敦诊所）；West Dermatology；University of Rijeka（里耶卡大学）；Clinical Hospital Center Rijeka（里耶卡临床医院中心）；LaserMed；Gazi University（加齐大学）；Institut Curie（居里研究所）；University of Paris（巴黎大学）；Universidad Autónoma de Madrid（马德里自治大学）；Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；Freie Universität Berlin（柏林自由大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；University Duisburg-Essen（杜伊斯堡-埃森大学）；University Hospital Essen（埃森大学医院）；Technische Universität Dresden（德累斯顿工业大学）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔兰根-纽伦堡大学）；Uniklinikum Erlangen（埃尔兰根大学医院）；University Hospital Regensburg（雷根斯堡大学医院）；University Hospital Würzburg（维尔茨堡大学医院）
Cross-Modal Benchmarking for Robotic Perception in Natural Environments
CSIRO Robotics, CSIRO, Australia（CSIRO机器人研究所，CSIRO，澳大利亚）；University of Sydney (USyd), Australia（悉尼大学（USyd），澳大利亚）；Queensland University of Technology (QUT), Australia（昆士兰理工大学（QUT），澳大利亚）
