Explainable Benchmarking through the Lense of Concept Learning
Faculty of Computer Science, Electrical Engineering and Mathematics（计算机科学与工程学院）；Data Science Group (DICE), Heinz Nixdorf Institute, Paderborn University（数据科学组（DICE）、海因茨·尼克斯多夫研究所、帕德博恩大学）
Quannian Zhang, Michael Röder, Nikit Srivastava, N'Dah Jean Kouagou, Axel-Cyrille Ngonga Ngomo
Evaluating competing systems in a comparable way, i.e., benchmarking them, is an undeniable pillar of the scientific method. However, system performance is often summarized via a small number of metrics. The analysis of the evaluation details and the derivation of insights for further development or use remains a tedious manual task with often biased results. Thus, this paper argues for a new type of benchmarking, which is dubbed explainable benchmarking. The aim of explainable benchmarking approaches is to automatically generate explanations for the performance of systems in a benchmark. We provide a first instantiation of this paradigm for knowledge-graph-based question answering systems. We compute explanations by using a novel concept learning approach developed for large knowledge graphs called PruneCEL. Our evaluation shows that PruneCEL outperforms state-of-the-art concept learners on the task of explainable benchmarking by up to 0.55 points F1 measure. A task-driven user study with 41 participants shows that in 80\% of the cases, the majority of participants can accurately predict the behavior of a system based on our explanations. Our code and data are available at https://github.com/dice-group/PruneCEL/tree/K-cap2025
The currently dominating artificial intelligence and machine learning technology, neural networks, builds on inductive statistical learning. Neural networks of today are information processing systems void of understanding and reasoning capabilities, consequently, they cannot explain promoted decisions in a humanly valid form. In this work, we revisit and use fundamental philosophy of science theories as an analytical lens with the goal of revealing, what can be expected, and more importantly, not expected, from methods that aim to explain decisions promoted by a neural network. By conducting …
B-XAIC Dataset: Benchmarking Explainable AI for Graph Neural Networks Using Chemical Data
University of Edinburgh（爱丁堡大学）；Jagiellonian University（雅盖隆大学）；Jagiellonian University, Faculty of Mathematics and Computer Science（雅盖隆大学数学与计算机科学学院）；Jagiellonian University, Faculty of Chemistry（雅盖隆大学化学学院）；Ardigen SA（Ardigen公司）
In this paper, we propose a gamification approach as a novel framework for smart building infrastructure with the goal of motivating human occupants to reconsider personal energy usage and to have positive effects on their environment. Human interaction in the context of cyber-physical systems is a core component and consideration in the implementation of any smart building technology. Research has shown that the adoption of human-centric building services and amenities leads to improvements in the operational efficiency of these cyber-physical systems directed towards controlling building ene…
SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions
Technical University of Munich, Helmholtz Munich, Munich Center for Machine Learning (MCML)（慕尼黑技术大学、亥姆霍兹慕尼黑、慕尼黑机器学习中心）；LTCI, Télécom Paris, Institut Polytechnique de Paris（LTCI、巴黎电信、巴黎理工学院）
Benchmarking Early Agitation Prediction in Community-Dwelling People with Dementia Using Multimodal Sensors and Machine Learning
KITE Research Institute（KITE研究机构）；Toronto Rehabilitation Institute（多伦多康复研究所）；University Health Network（大学健康网络）；Lawrence Bloomberg Faculty of Nursing（劳伦斯布罗德利护理学院）；University of Toronto（多伦多大学）；College of Engineering and Technology（工程与技术学院）；American University of the Middle East（中东美国大学）
Explainable Fuzzy GNNs for Leak Detection in Water Distribution Networks
Jheronimus Academy of Data Science, Eindhoven University of Technology, Eindhoven, The Netherlands University of Bari Aldi Moro, Bari, Italy IHE Institute for Water Education, Delft, The Netherlands University of Cartagena, Cartagena, Spain School of Industrial Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands
Measuring What Matters: A Unified Evaluation Framework for GNN Explainability
Sapienza University of Rome（罗马萨皮恩扎大学）；Intesa Sanpaolo AI Research（Intesa Sanpaolo人工智能研究）；Anti Financial Crime Digital Hub（反金融犯罪数字中心）；Intesa Sanpaolo Innovation Center（Intesa Sanpaolo创新中心）
SPINEX-TimeSeries: Similarity-based Predictions with Explainable Neighbors Exploration for Time Series and Forecasting Problems
University of Manitoba（曼尼托巴大学）；Clemson University（克莱姆森大学）；Artificial Intelligence Research Institute for Science and Engineering (AIRISE), Clemson University（克莱姆森大学科学与工程人工智能研究院（AIRISE））
Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI
Aristotle University of Thessaloniki, School of Informatics（塞萨洛尼基亚里士多德大学信息学院）；Technical University of Denmark（丹麦技术大学）；Telefonica Scientific Research（电信科学研究院）
Classifying nodes in knowledge graphs is an important task, e.g., for predicting missing types of entities, predicting which molecules cause cancer, or predicting which drugs are promising treatment candidates. While black-box models often achieve high predictive performance, they are only post-hoc and locally explainable and do not allow the learned model to be easily enriched with domain knowledge. Towards this end, learning description logic concepts from positive and negative examples has been proposed. However, learning such concepts often takes a long time and state-of-the-art approaches…
