Artificial intelligence (AI) is currently based largely on black-box machine learning models which lack interpretability. The field of eXplainable AI (XAI) strives to address this major concern, being critical in high-stakes areas such as the finance, legal and health sectors.
We present an approach to defining AI models and their interpretability based on category theory. For this we employ the notion of a compositional model, which sees a model in terms of formal string diagrams which capture its abstract structure together with its concrete implementation. This comprehensive view incorporates deterministic, probabilistic and quantum models. We compare a wide range of AI models as compositional models, including linear and rule-based models, (recurrent) neural networks, transformers, VAEs, and causal and DisCoCirc models.
Next we give a definition of interpretation of a model in terms of its compositional structure, demonstrating how to analyse the interpretability of a model, and using this to clarify common themes in XAI. We find that what makes the standard 'intrinsically interpretable' models so transparent is brought out most clearly diagrammatically. This leads us to the more general notion of compositionally-interpretable (CI) models, which additionally include, for instance, causal, conceptual space, and DisCoCirc models.
We next demonstrate the explainability benefits of CI models. Firstly, their compositional structure may allow the computation of other quantities of interest, and may facilitate inference from the model to the modelled phenomenon by matching its structure. Secondly, they allow for diagrammatic explanations for their behaviour, based on influence constraints, diagram surgery and rewrite explanations. Finally, we discuss many future directions for the approach, raising the question of how to learn such meaningfully structured models in practice.
Black Box Model Explanations and the Human Interpretability Expectations -- An Analysis in the Context of Homicide Prediction
Federal University of Pará (UFPA)(帕拉联邦大学);Federal Institute of Education, Science and Technology of Pará (IFPA)(帕拉联邦教育、科学和技术学院);Vale Institute of Technology (ITV DS)(淡水河谷技术研究院)
What Does Evaluation of Explainable Artificial Intelligence Actually Tell Us? A Case for Compositional and Contextual Validation of XAI Building Blocks
AI Readiness in Healthcare through Storytelling XAI
Center for Artificial Intelligence in Public Health Research (ZKI-PH) at Robert Koch Institute(罗伯特·科赫研究所公共卫生研究人工智能中心);Free University of Berlin(柏林自由大学)
针对医疗 AI 落地中的信任与差异化解释需求,本文提出结合多任务蒸馏和可解释性技术的叙事式 XAI,以支持复杂深度模型并提升专家信任。
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)(电气工程与计算机科学学院(SEECS),国家 sciences and Technology(NUST))
Explainable Artificial Intelligence (XAI) on TimeSeries Data: A Survey
ENSTA Paris, Institut Polytechnique Paris(巴黎高等先进技术学院,巴黎综合理工学院);Inria Flowers(法国国家信息与自动化研究所Flowers团队);Renault(雷诺公司);TECNALIA(TECNALIA研究中心);Basque Research and Technology Alliance (BRTA)(巴斯克研究与技术联盟);University of the Basque Country (UPV/EHU)(巴斯克大学)
Bridging Accuracy and Interpretability: Deep Learning with XAI for Breast Cancer Detection
Department of Mathematics and Statistics, Indian Institute of Technology Kanpur, Kanpur - 208016, Uttar Pradesh, India(数学与统计学系,印度理工学院坎浦尔学院,坎浦尔 - 208016,印度乌塔兰普尔州)
A Comprehensive Guide to Explainable AI: From Classical Models to LLMs
National Tsing Hua University(国立清华大学);Indiana University(印第安纳大学);National University of Singapore(新加坡国立大学);Kyoto University(京都大学);AppCubic;Rutgers University(罗格斯大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Purdue University(普渡大学);Georgia Institute of Technology(佐治亚理工学院);National Taiwan Normal University(国立台湾师范大学);University of Hawaii(夏威夷大学);Xi’an Jiaotong-Liverpool University(西交利物浦大学);Pingtan Research Institute of Xiamen University(厦门大学平潭研究院);University of Manchester(曼彻斯特大学);Aarhus University(奥胡斯大学);Imperial College London(伦敦帝国学院);School of Visual Arts(视觉艺术学院);The University of Texas at Dallas(德克萨斯大学达拉斯分校)
Applications of Explainable artificial intelligence in Earth system science
Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)(南方海洋科学与工程广东省实验室(珠海));Guangdong Province Key Laboratory for Climate Change and Natural Disaster Studies(广东省气候变化与自然灾害研究重点实验室);Max Planck Institute for Biogeochemistry(马克斯·普朗克生物地球化学研究所);Shanwei Meteorological Service(汕尾市气象局);College of Computer Science and Technology, Changchun Normal University(长春师范大学计算机科学与技术学院);ELLIS Unit Jena(ELLIS耶拿单元)
RetinaVision: XAI-Driven Augmented Regulation for Precise Retinal Disease Classification using deep learning framework
East West University(东西大学);American International University-Bangladesh(孟加拉国美国国际大学);Bangladesh Army University of Engineering and Technology(孟加拉国陆军工程技术大学)
Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classification
ENSTA, Institut Polytechnique Paris(巴黎理工学院ENSTA);Inria(法国国家信息与自动化研究所);Segula Technologies(赛科集团);Sorbonne Université(索邦大学);TECNALIA(泰克尼亚研究中心);University of the Basque Country (UPV/EHU)(巴斯克大学);Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI)(安达卢西亚数据科学与计算智能研究所);University of Granada(格拉纳达大学)
A Novel XAI-Enhanced Quantum Adversarial Networks for Velocity Dispersion Modeling in MaNGA Galaxies
Oracle Financial Services Software Ltd.(Oracle金融服务软件有限公司);Indian Institute of Information Technology Dharwad(德瓦德信息学院);NVIDIA;Google AI(谷歌人工智能);Apple AI(苹果人工智能)