The black-box nature of deep learning models prevents them from being completely trusted in domains like biomedicine. Most explainability techniques do not capture the concept-based reasoning that human beings follow. In this work, we attempt to understand the behavior of trained models that perform image processing tasks in the medical domain by building a graphical representation of the concepts they learn. Extracting such a graphical representation of the model's behavior on an abstract, higher conceptual level would unravel the learnings of these models and would help us to evaluate the steps taken by the model for predictions. We show the application of our proposed implementation on two biomedical problems - brain tumor segmentation and fundus image classification. We provide an alternative graphical representation of the model by formulating a concept level graph as discussed above, which makes the problem of intervention to find active inference trails more tractable. Understanding these trails would provide an understanding of the hierarchy of the decision-making process followed by the model. [As well as overall nature of model]. Our framework is available at https://github.com/koriavinash1/BioExp
Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Framework
Peking University(北京大学);Harbin Institute of Technology(哈尔滨工业大学);The Hong Kong University of Science and Technology(香港科技大学);Harbin Institute of Technology (Weihai)(哈尔滨工业大学(威海))
UCSF(加州大学旧金山分校);SFVAMC(旧金山退伍军人事务医疗中心);UC Berkeley(加州大学伯克利分校);Helen Diller Family Comprehensive Cancer Center, UCSF(加州大学旧金山分校海伦·迪勒家族综合癌症中心);Bakar Computational Health Sciences Institute, UCSF(加州大学旧金山分校巴卡尔计算健康科学研究所)
SPARC: Concept-Aligned Sparse Autoencoders for Cross-Model and Cross-Modal Interpretability
Department of Computer Science and Software Engineering (CSSE) Concordia University, Canada(计算机科学与软件工程系(CSSE)康科迪亚大学,加拿大);Department of Electrical and Computer Engineering (ECE) Concordia University, Canada(电气与计算机工程系(ECE)康科迪亚大学,加拿大)
Exploring Interpretability for Visual Prompt Tuning with Cross-layer Concepts
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Microsoft Research Asia(微软亚洲研究院);School of Telecommunication and Engineering, Xidian University(西安电子科技大学电信与工程学院)
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
The Berlin Institute for the Foundations of Learning and Data (BIFOLD)(柏林学习与数据基础研究所);Technische Universität Berlin(柏林工业大学);INRAE(法国国家农业、食品与环境研究院);Inria, EVERGREEN(法国国家信息与自动化研究所,EVERGREEN);UMR TETIS, Univ Montpellier(UMR TETIS,蒙彼利埃大学)
CBM-RAG: Demonstrating Enhanced Interpretability in Radiology Report Generation with Multi-Agent RAG and Concept Bottleneck Models
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Saarland University(萨尔兰大学);University of Oldenburg(奥尔登堡大学);Dhaka Medical College Hospital(达卡医学院医院)
Attribute-formed Class-specific Concept Space: Endowing Language Bottleneck Model with Better Interpretability and Scalability
University of Electronic Science and Technology of China(电子科技大学);Southwest Jiaotong University(西南交通大学);University of Minnesota(明尼苏达大学);Harvard University(哈佛大学);Nanyang Technological University(南洋理工大学)
ICICLE: Interpretable Class Incremental Continual Learning
Jagiellonian University(雅盖隆大学);Ardigen SA(阿迪根公司);Autonomous University of Barcelona(巴塞罗那自治大学);Computer Vision Center(计算机视觉中心);IDEAS NCBR(IDEAS国家计算智能研究中心)
ConceptHash: Interpretable Fine-Grained Hashing via Concept Discovery
University of Surrey(萨里大学);iFlyTek-Surrey Joint Research Centre on Artificial Intelligence(讯飞-萨里人工智能联合研究中心);Surrey Institute for People-Centred Artificial Intelligence(萨里以人为本人工智能研究所);CVSSP(视觉、语音和信号处理中心)
Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation
Department of Engineering(工程系);Research Unit of Artificial Intelligence and Computer Systems(人工智能与计算机系统研究单位);Università Campus Bio-Medico of Roma(罗马大学生物医学校园);Department of Computer, Control and Management Engineering(计算机、控制与管理工程系);Sapienza University of Rome(罗马萨皮恩扎大学);Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering(诊断与介入、辐射物理、生物医学工程系);Umeå University(乌梅拉大学);UniCamillus-Saint Camillus International University of Health Sciences(UniCamillus-圣卡米卢斯国际健康科学大学)
Technical University of Kaiserslautern(凯泽斯劳滕工业大学);Technical University of Berlin(柏林工业大学);University of Birmingham(伯明翰大学);NPZ Innovation GmbH(NPZ创新有限公司);University of St.Gallen(圣加仑大学)
Learning Concept-Driven Logical Rules for Interpretable and Generalizable Medical Image Classification
School of Data Science, Fudan University(复旦大学数据科学学院);Dept. of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程系);Dept. of Oncology, University of Cambridge(剑桥大学肿瘤学系)
Towards Multi-dimensional Explanation Alignment for Medical Classification
King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);HKUST(GZ)(香港科技大学(广州));Tongji University(同济大学);Ant Group(蚂蚁集团);The University of Auckland(奥克兰大学);RIKEN Center for Advanced Intelligence Project (AIP)(理化学研究所先进智能项目中心)
Concept-Guided Interpretability via Neural Chunking
Allen Institute(阿伦研究所);University of Washington(华盛顿大学);Télécom Paris, Institut Polytechnique de Paris(巴黎高等电信学院);Institute of Explainable Machine Learning, Helmholtz Munich(可解释机器学习研究所,海德堡大学);Department of Computational Neuroscience, Max Planck Institute for Biological Cybernetics(生物信息学研究所,马克斯·普朗克研究院);Institute for Human-Centered AI, Helmholtz Munich(以人为中心的人工智能研究所,海德堡大学)
On Interpretability of Deep Learning based Skin Lesion Classifiers using Concept Activation Vectors
Hochschule Pforzheim(普福尔茨海姆应用技术大学);Smart Data and Knowledge Services German Research Center for Artificial Intelligence GmbH (DFKI)(德国人工智能研究中心智能数据与知识服务部);Technische Universität Kaiserslautern(凯泽斯劳滕工业大学);Universitätsklinikum Münster(明斯特大学医院);Universitätsklinikum Düsseldorf(杜塞尔多夫大学医院);National University of Science and Technology (NUST)(巴基斯坦国立科技大学);National Center of Artificial Intelligence(国家人工智能中心)
Chat-CBM: Towards Interactive Concept Bottleneck Models with Frozen Large Language Models
Department of Biomedical Engineering, College of Future Technology, Peking University(生物医学工程系,未来技术学院,北京大学);Institute of Medical Technology, Peking University Health Science Center, Peking University(医学技术研究所,北京大学医学部,北京大学);National Biomedical Imaging Center, College of Future Technology, Peking University(国家生物医学成像中心,未来技术学院,北京大学)
Data and Web Science Group, University of Mannheim, Mannheim, Germany(曼海姆大学数据与网络科学小组);Clausthal University of Technology, Clausthal, Germany(克虏格大学);Max-Planck-Institute for Informatics, Saarland Informatics Campus(马克斯·普朗克信息研究所)