Information Bottleneck-based Causal Attention for Multi-label Medical Image Recognition
Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR)（联合山东大学-东北大学人工智能研究中心（C-FAIR））；Shandong University（山东大学）；School of Information Science and Engineering（信息科学与工程学院）；Child Care Hospital Affiliated to Shandong First Medical University（山东第一医科大学附属儿童医院）；Case Western Reserve University（凯斯西储大学）
Xiaoxiao Cui, Yiran Li, Kai He, Shanzhi Jiang, Mengli Xue, Wentao Li, Junhong Leng, Zhi Liu, Lizhen Cui, Shuo Li
Multi-label classification (MLC) of medical images aims to identify multiple diseases and holds significant clinical potential. A critical step is to learn class-specific features for accurate diagnosis and improved interpretability effectively. However, current works focus primarily on causal attention to learn class-specific features, yet they struggle to interpret the true cause due to the inadvertent attention to class-irrelevant features. To address this challenge, we propose a new structural causal model (SCM) that treats class-specific attention as a mixture of causal, spurious, and noisy factors, and a novel Information Bottleneck-based Causal Attention (IBCA) that is capable of learning the discriminative class-specific attention for MLC of medical images. Specifically, we propose learning Gaussian mixture multi-label spatial attention to filter out class-irrelevant information and capture each class-specific attention pattern. Then a contrastive enhancement-based causal intervention is proposed to gradually mitigate the spurious attention and reduce noise information by aligning multi-head attention with the Gaussian mixture multi-label spatial. Quantitative and ablation results on Endo and MuReD show that IBCA outperforms all methods. Compared to the second-best results for each metric, IBCA achieves improvements of 6.35\% in CR, 7.72\% in OR, and 5.02\% in mAP for MuReD, 1.47\% in CR, and 1.65\% in CF1, and 1.42\% in mAP for Endo.
Statistical machine learning algorithms have achieved state-of-the-art results on benchmark datasets, outperforming humans in many tasks. However, the out-of-distribution data and confounder, which have an unpredictable causal relationship, significantly degrade the performance of the existing models. Causal Representation Learning (CRL) has recently been a promising direction to address the causal relationship problem in vision understanding. This survey presents recent advances in CRL in vision. Firstly, we introduce the basic concept of causal inference. Secondly, we analyze the CRL theoret…
Imperial College London（帝国理工学院）；TU München（慕尼黑工业大学）；FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
Medical image analysis is a vibrant research area that offers doctors and medical practitioners invaluable insight and the ability to accurately diagnose and monitor disease. Machine learning provides an additional boost for this area. However, machine learning for medical image analysis is particularly vulnerable to natural biases like domain shifts that affect algorithmic performance and robustness. In this paper we analyze machine learning for medical image analysis within the framework of Technology Readiness Levels and review how causal analysis methods can fill a gap when creating robust…
Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis
Hong Kong University of Science and Technology（香港科技大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Chemical and Biological Engineering（化学与生物工程系）；Division of Life Science（生命科学部）
Huazhong University of Science and Technology（华中科技大学）；Guizhou Normal University（贵州师范大学）；Hubei University（湖北大学）
Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectivel…
This article discusses how the language of causality can shed new light on the major challenges in machine learning for medical imaging: 1) data scarcity, which is the limited availability of high-quality annotations, and 2) data mismatch, whereby a trained algorithm may fail to generalize in clinical practice. Looking at these challenges through the lens of causality allows decisions about data collection, annotation procedures, and learning strategies to be made (and scrutinized) more transparently. We discuss how causal relationships between images and annotations can not only have profound…
The Boundaries of Fair AI in Medical Image Prognosis: A Causal Perspective
Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学）；Department of Biomedical Informatics, The Ohio State University（生物医学信息学系，俄亥俄州立大学）；Department of Ophthalmology and Visual Sciences, The Ohio State University（眼科学与视觉科学系，俄亥俄州立大学）
Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts
CIBM Center for Biomedical Imaging（生物医学成像中心）；Department of Radiology, Lausanne University Hospital and University of Lausanne（拉沃大学医院放射科）；MedGIFT, Institute of Informatics, School of Management, HES–SO Valais–Wallis University of Applied Sciences and Arts Western Switzerland（MedGIFT，信息学院，管理学院，瓦莱-瓦莱大学应用科学和艺术西瑞士）
MVP-CBM:Multi-layer Visual Preference-enhanced Concept Bottleneck Model for Explainable Medical Image Classification
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（中国科学技术大学生物医学工程学院、生命科学与医学系）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏等候研院）；Jiangsu Provincial Key Laboratory of Multimodal Digital Twin Technology（江苏省多模态数字孪生技术省级重点实验室）；Medical Business Department, iFlytek Co., Ltd（iFlytek有限公司医疗业务部）；State Key Laboratory of Precision and Intelligent Chemistry, USTC（精准与智能化学国家重点实验室）
Towards Data-Efficient Medical Imaging: A Generative and Semi-Supervised Framework
Dept. of Electrical and Electronic Engineering Imperial College London（电气与电子工程系帝国理工学院伦敦）；Centre for Vision, Speech and Signal Processing University of Surrey（视觉、语音和信号处理中心萨里大学）
MIMM-X: Disentangling Spurious Correlations for Medical Image Analysis
Medical Image and Data Analysis, University Hospital of Tübingen, Germany（医学图像与数据分析，图宾根大学医院，德国）；Institute for Signal Processing and System Theory, University of Stuttgart, Germany（信号处理与系统理论研究所，斯图加特大学，德国）；Stanford University, Department of Radiology, Stanford, USA（斯坦福大学放射科，斯坦福，美国）
Imperial College London（帝国理工学院）；FAU Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
Curating a large scale medical imaging dataset for machine learning applications is both time consuming and expensive. Balancing the workload between model development, data collection and annotations is difficult for machine learning practitioners, especially under time constraints. Causal analysis is often used in medicine and economics to gain insights about the effects of actions and policies. In this paper we explore the effect of dataset interventions on the output of image classification models. Through a causal approach we investigate the effects of the quantity and type of data we nee…
UNet with Self-Adaptive Mamba-Like Attention and Causal-Resonance Learning for Medical Image Segmentation
Faculty of Computing and IT (FCIT)（计算与信息技术学院）；Sohar University（索哈尔大学）；Department of Intelligent Systems（智能系统系）；KTH Royal Institute of Technology（皇家理工学院）；College of Computer and Information Sciences（计算机与信息科学学院）；Imam Mohammad Ibn Saud Islamic University (IMSIU)（伊玛目穆罕默德·本·沙特伊斯兰大学）；University of Auckland（奥克兰大学）；Department of Computer Science, Faculty of Science（科学学院计算机科学系）；Northern Border University（北方边境大学）
Scientific Computing and Imaging Institute（科学计算与成像研究所）；University of Utah（犹他大学）
Statistical shape modeling (SSM) directly from 3D medical images is an underutilized tool for detecting pathology, diagnosing disease, and conducting population-level morphology analysis. Deep learning frameworks have increased the feasibility of adopting SSM in medical practice by reducing the expert-driven manual and computational overhead in traditional SSM workflows. However, translating such frameworks to clinical practice requires calibrated uncertainty measures as neural networks can produce over-confident predictions that cannot be trusted in sensitive clinical decision-making. Existin…
CausalCLIPSeg: Unlocking CLIP's Potential in Referring Medical Image Segmentation with Causal Intervention
Wuhan University of Technology（武汉理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；MedAI Technology (Wuxi) Co. Ltd.（MedAI科技（无锡）有限公司）；Technical University of Munich（慕尼黑工业大学）
University of Florida（佛罗里达大学）；Dell EMC（戴尔易安信）；Intel（英特尔）
Using medical imaging as case-study, we demonstrate how Intel-optimized TensorFlow on an x86-based server equipped with 2nd Generation Intel Xeon Scalable Processors with large system memory allows for the training of memory-intensive AI/deep-learning models in a scale-up server configuration. We believe our work represents the first training of a deep neural network having large memory footprint (~ 1 TB) on a single-node server. We recommend this configuration to scientists and researchers who wish to develop large, state-of-the-art AI models but are currently limited by memory.
Structure Causal Models and LLMs Integration in Medical Visual Question Answering
School of Microelectronics, Tianjin University（天津大学微电子学院）；School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）
Causal Disentanglement for Robust Long-tail Medical Image Generation
School of Electrical and Information Engineering, Tianjin University, China（天津大学电气与信息工程学院）；Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；Fondazione Bruno Kessler, Italy（布鲁诺·克塞勒基金会）
Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation
City University of Hong Kong（香港城市大学）；Shenzhen Loop Area Institute（深圳河套学院）；CAIR, HKISI, Chinese Academy of Sciences（中国科学院计算智能研究所）；UESTC（电子科技大学）；MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Models
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）；Shenzhen Polytechnic University（深圳职业技术大学）；University of Macau（澳门大学）；Southern University of Science and Technology（南方科技大学）
MAMBO-NET: Multi-Causal Aware Modeling Backdoor-Intervention Optimization for Medical Image Segmentation Network
Tianjin University（天津大学）；Tianjin Key Laboratory of Cognitive Computing and Application（天津认知计算与应用重点实验室）；Tianjin Key Laboratory of Advanced Networking（天津先进网络重点实验室）；Tianjin International Engineering Institute, Tianjin University（天津大学国际工程师学院）；Tianjin Central Hospital of Gynecology Obstetrics（天津市中心妇产科医院）；Silesian University of Technology（西里西亚工业大学）；University of Alberta（阿尔伯塔大学）；Istinye University（伊斯坦布尔耶尼大学）
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）；Deepwise AI Lab（深睿医疗AI实验室）
We propose a causal hidden Markov model to achieve robust prediction of irreversible disease at an early stage, which is safety-critical and vital for medical treatment in early stages. Specifically, we introduce the hidden variables which propagate to generate medical data at each time step. To avoid learning spurious correlation (e.g., confounding bias), we explicitly separate these hidden variables into three parts: a) the disease (clinical)-related part; b) the disease (non-clinical)-related part; c) others, with only a),b) causally related to the disease however c) may contain spurious co…
Dual Causal Inference: Integrating Backdoor Adjustment and Instrumental Variable Learning for Medical VQA
School of Microelectronics, Tianjin University（天津大学微电子学院）；Department of Orthopedics, Affiliated Kunshan Hospital of Jiangsu University（江苏大学附属昆山医院骨科部）；Department of Clinical Laboratory, The Third Central Hospital of Tianjin（天津第三中心医院临床实验室）；School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）
Deep convolutional neural networks have been proven to be very effective in image related analysis and tasks, such as image segmentation, image classification, image generation, etc. Recently many sophisticated CNN based architectures have been proposed for the purpose of image segmentation. Some of these newly designed networks are used for the specific purpose of medical image segmentation, models like V-Net, U-Net and their variants. It has been shown that U-Net produces very promising results in the domain of medical image segmentation.However, in this paper, we argue that the architecture…
Cross-Modal Causal Intervention for Medical Report Generation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）；School of Science, Sun Yat-sen University（中山大学理学院）；Hong Kong Polytechnic University（香港理工大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
