Complementary Information Mutual Learning for Multimodality Medical Image Segmentation
School of Computer Science and Technology(计算机科学与技术学院);East China Normal University(华东师范大学);School of Data Science(数据科学学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳市人工智能与机器人研究院);Huashan Hospital(华山医院);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);School of Software Engineering(软件工程学院);Shanghai Research Institute for Intelligent Autonomous Systems(上海智能自主系统研究院);Tongji University(同济大学)
Chuyun Shen, Wenhao Li, Haoqing Chen, Xiaoling Wang, Fengping Zhu, Yuxin Li, Xiangfeng Wang, Bo Jin
Radiologists must utilize multiple modal images for tumor segmentation and diagnosis due to the limitations of medical imaging and the diversity of tumor signals. This leads to the development of multimodal learning in segmentation. However, the redundancy among modalities creates challenges for existing subtraction-based joint learning methods, such as misjudging the importance of modalities, ignoring specific modal information, and increasing cognitive load. These thorny issues ultimately decrease segmentation accuracy and increase the risk of overfitting. This paper presents the complementary information mutual learning (CIML) framework, which can mathematically model and address the negative impact of inter-modal redundant information. CIML adopts the idea of addition and removes inter-modal redundant information through inductive bias-driven task decomposition and message passing-based redundancy filtering. CIML first decomposes the multimodal segmentation task into multiple subtasks based on expert prior knowledge, minimizing the information dependence between modalities. Furthermore, CIML introduces a scheme in which each modality can extract information from other modalities additively through message passing. To achieve non-redundancy of extracted information, the redundant filtering is transformed into complementary information learning inspired by the variational information bottleneck. The complementary information learning procedure can be efficiently solved by variational inference and cross-modal spatial attention. Numerical results from the verification task and standard benchmarks indicate that CIML efficiently removes redundant information between modalities, outperforming SOTA methods regarding validation accuracy and segmentation effect.
Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentation
University of Copenhagen(哥本哈根大学);ETH Zurich(苏黎世联邦理工学院);Nankai University(南开大学);University of Oxford(牛津大学);Institute for Infocomm Research, A*STAR(新加坡科技研究局信息通信研究所)
Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Northwestern Polytechnical University(西北工业大学);Inception Institute of Artificial Intelligence(因塞普申人工智能研究院)
M$^3$HL: Mutual Mask Mix with High-Low Level Feature Consistency for Semi-Supervised Medical Image Segmentation
Shanghai Key Laboratory of Intelligent Sensing and Recognition(上海智能感知与识别重点实验室);Shanghai Jiao Tong University(上海交通大学);Department of Endocrinology and Metabolism, Renji Hospital, School of Medicine, Shanghai Jiao Tong University(内分泌与代谢科,仁济医院,医学院,上海交通大学);Center for Digital Innovation, Tongji University(同济大学数字创新中心)
Mutual Evidential Deep Learning for Medical Image Segmentation
Key Laboratory of High Confidence Software Technologies (MOE) School of Computer Science Peking University(高可信软件技术重点实验室(MOE)计算机科学学院,北京大学);College of Computer and Information Science School of Software Southwest University(计算机与信息科学学院 软件学院 西南大学);Department of Computer and Information Science University of Macau(计算机与信息科学系 澳门大学)
Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation
Zhejiang University(浙江大学);Zhejiang Gongshang University(浙江工商大学);Jilin University(吉林大学);National University of Singapore(新加坡国立大学);Black Sesame Technologies(黑芝麻科技)
S2ML: Spatio-Spectral Mutual Learning for Depth Completion
Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Tsinghua University(数据与信息研究所,清华大学深圳国际研究生院,清华大学);School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University(电子工程系和通信工程研究所,国立清华大学)
Robust Brain Tumor Segmentation with Incomplete MRI Modalities Using Hölder Divergence and Mutual Information-Enhanced Knowledge Transfer
MSU-BIT-SMBU Joint Research Center of Applied Mathematics, Shenzhen MSU-BIT University(深圳MSU-BIT大学应用数学联合研究中心);Institute of Control Theory and Control Engineering, School of Automation, Beijing Institute of Technology(北京理工大学控制理论与控制工程研究所);School of Mathematics and Statistics, Beijing Institute of Technology(北京理工大学数学与统计学学院);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东人工智能与数字经济实验室(深圳))
Band Selection and Classification of Hyperspectral Images using Mutual Information: An algorithm based on minimizing the error probability using the inequality of Fano
Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images
Southeast University(东南大学);Centre de Recherche en Information Biomédicale Sino-Français (CRIBs)(中法生物医学信息研究中心);Univ Rennes(雷恩大学);Inserm(法国国家健康与医学研究院);LTSI - UMR1099(信号与图像处理实验室)
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)