Applying network science approaches to investigate the functions and anatomy of the human brain is prevalent in modern medical imaging analysis. Due to the complex network topology, for an individual brain, mining a discriminative network representation from the multimodal brain networks is non-trivial. The recent success of deep learning techniques on graph-structured data suggests a new way to model the non-linear cross-modality relationship. However, current deep brain network methods either ignore the intrinsic graph topology or require a network basis shared within a group. To address these challenges, we propose a novel end-to-end deep graph representation learning (Deep Multimodal Brain Networks - DMBN) to fuse multimodal brain networks. Specifically, we decipher the cross-modality relationship through a graph encoding and decoding process. The higher-order network mappings from brain structural networks to functional networks are learned in the node domain. The learned network representation is a set of node features that are informative to induce brain saliency maps in a supervised manner. We test our framework in both synthetic and real image data. The experimental results show the superiority of the proposed method over some other state-of-the-art deep brain network models.
Federated Modality-specific Encoders and Partially Personalized Fusion Decoder for Multimodal Brain Tumor Segmentation
National Institute for Data Science in Health and Medicine（国家医学数据科学研究院）；Department of Computer Science at School of Informatics（信息学院计算机科学系）；Jarvis Research Center（Jarvis研究中心）；Medical Artificial Intelligence Lab（医学人工智能实验室）
Large-Scale Pre-training Enables Multimodal AI Differentiation of Radiation Necrosis from Brain Metastasis Progression on Routine MRI
University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根大学医院，埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN (CCC ER-EMN)（埃尔朗根-欧洲大都会区纽伦堡综合癌症中心）；Bavarian Cancer Research Center (BZKF)（巴伐利亚癌症研究中心）；Universität Würzburg（维尔茨堡大学）；Peking University（北京大学）；Universitätsklinikum Würzburg（维尔茨堡大学医院）
Towards Label-Free Brain Tumor Segmentation: Unsupervised Learning with Multimodal MRI
Universitat Politècnica de Catalunya (UPC)（西班牙巴塞罗那理工大学）；University College Dublin (UCD)（都柏林大学）；Dublin City University (DCU)（都柏林城市大学）；Insight Research Ireland Center For Data Analytics（爱尔兰数据分析研究中心）
Fourier Disentangled Multimodal Prior Knowledge Fusion for Red Nucleus Segmentation in Brain MRI
Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎大脑研究所（ICM））；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共援助医院集团）；Hôpital de la Pitié Salpêtrière（皮蒂耶-萨尔佩特里埃医院）
Modality-Agnostic Input Channels Enable Segmentation of Brain lesions in Multimodal MRI with Sequences Unavailable During Training
Department of Engineering Science, University of Oxford, Oxford, UK（工程科学系，牛津大学，牛津，英国）；Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK（牛津大学临床神经科学系，牛津，英国）
Instituto de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València（瓦伦西亚理工大学先进信息技术与通信应用研究所 (ITACA)）；CNRS（法国国家科学研究中心）；Univ. Bordeaux（波尔多大学）；Bordeaux INP（波尔多国立高等工程技术学校）；Universitat de Valencia（瓦伦西亚大学）；University Valencia（瓦伦西亚大学）；EPHE（巴黎高等研究实践学院）；CHU de Bordeaux（波尔多大学医院中心）；Institut des Maladies Neurodégénératives（神经退行性疾病研究所）；CEA Saclay（法国原子能和替代能源委员会萨克雷）；Instituto de Automática e Informática Industrial, Universitat Politècnica de València（瓦伦西亚大学工业自动化与计算机研究所）；Hospital Universitario y Politécnico La Fe（拉菲大学综合医院）；Fundación para el Fomento de la Investigación Sanitario y Biomédica de la Comunidad Valenciana（瓦伦西亚社区健康与生物医学研究促进基金会）
Faculty of Computer Science, AGH University of Krakow（计算机科学系，克拉科夫AGH大学）；Department of Informatics, Universitas Pembangunan Nasional Veteran Yogyakarta（信息系，全国 veterans 大学 Yogya 市）
Multiplex Graph Networks for Multimodal Brain Network Analysis
Lehigh University（理海大学）；Beihang University（北京航空航天大学）；University of Pittsburgh（匹兹堡大学）；Perelman School of Medicine（佩雷尔曼医学院）；University of Pennsylvania（宾夕法尼亚大学）
Characterization Multimodal Connectivity of Brain Network by Hypergraph GAN for Alzheimer's Disease Analysis
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University（深圳大学）；Renmin University of China（中国人民大学）；Harbin Engineering University（哈尔滨工程大学）
Scalable multimodal convolutional networks for brain tumour segmentation
University College London（伦敦大学学院）；University College London Hospitals（伦敦大学学院医院）；Wellcome/EPSRC Centre for Surgical and Interventional Science, UCL（惠康/EPSRC 外科与介入科学中心，伦敦大学学院）
A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli
Beijing Institute of Basic Medical Sciences（北京基础医学研究所）；School of Computer Science and Information Engineering, Hefei University of Technology (HFUT)（合肥工业大学计算机与信息工程学院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Zhejiang University（浙江大学）；Australian Artificial Intelligence Institute, Faculty of Engineering and Information Technology, University of Technology Sydney（悉尼科技大学工程与信息技术学院澳大利亚人工智能研究所）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）
Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion
The Chinese University of Hong Kong（香港中文大学）；Imperial College Longdon（伦敦帝国理工学院）；Imsight Medical Technology Co. Ltd.（艾迈医疗科技有限公司）；The Hong Kong Polytechnic University（香港理工大学）；Chinese Academy of Sciences（中国科学院）
Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval
Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology（武汉理工大学交通物联网湖北省重点实验室）；State Key Laboratory for Multimedia Information Processing, Peking University（北京大学多媒体信息处理全国重点实验室）
Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning
Università Politecnica delle Marche（马尔凯理工大学）；University of Macerata（马切拉塔大学）；Horizon Intelligence Labs（地平线智能实验室）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
Image-to-Brain Signal Generation for Visual Prosthesis with CLIP Guided Multimodal Diffusion Models
Jinan University, Guangzhou, China；Department of Rehabilitation Medicine, The First Affiliated Hospital of Jinan University, Guangzhou, China；Western University, Ontario, Canada；Guangdong University of Technology, Guangzhou, China；Tsinghua University, Beijing, China
Multimodal Volume-Aware Detection and Segmentation for Brain Metastases Radiosurgery
Massachusetts General Hospital（麻省总医院）；Massachusetts Institute of Technology（麻省理工学院）；National Taiwan University Hospital（台湾大学医院）；Vysioneer Inc.（Vysioneer公司）
mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tencent（腾讯）；University of California, Los Angeles（加州大学洛杉矶分校）；Lenovo Research（联想研究院）
BrainMAP: Multimodal Graph Learning For Efficient Brain Disease Localization
School of Computing Technologies, RMIT University（计算技术学院，拉筹纳斯大学）；Institute of Innovation, Science and Sustainability, Federation University Australia（创新、科学与可持续性研究所，联邦大学澳大利亚）
Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume
Chinese PLA General Hospital（中国人民解放军总医院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；Beijing Jiaotong University（北京交通大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Probing Multimodal Fusion in the Brain: The Dominance of Audiovisual Streams in Naturalistic Encoding
Distributed and Intelligent Optimization Research Laboratory（分布式智能优化研究实验室）；Electrical Engineering Department（电气工程系）；Amirkabir University of Technology（阿米尔卡比尔技术大学）
H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；Northwestern Polytechnical University（西北工业大学）；University of Pittsburgh（匹兹堡大学）；University of Sydney（悉尼大学）；JD Finance America Corporation（京东金融美国公司）
The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge
Inria Saclay Île-de-France（法国里昂萨克利研究所）；CEA（法国原子能委员会）；Université Paris-Saclay（巴黎-萨克利大学）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化技术研究所）；Inserm（法国国家医学研究院）；AP-HP（法国国家医院集团）；Hôpital de la Pitié Salpêtrière（皮蒂埃-萨尔普里埃尔医院）；Harvard Medical School（哈佛医学院）；Brigham and Women's Hospital（布里根医院）；Department of Radiology and Radiological Science（放射学与放射科学系）；Johns Hopkins Medical School（约翰霍普金斯医学院）；Institute of Medical Informatics（医学信息学研究所）；University of Lübeck（吕贝克大学）；Image Analysis and Communications Laboratory（图像分析与通信实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Johns Hopkins University（约翰霍普金斯大学）；CSAIL（媒体实验室）
UPMAD-Net: A Brain Tumor Segmentation Network with Uncertainty Guidance and Adaptive Multimodal Feature Fusion
School of Computer Science and Technology, Zhengzhou University of Light Industry（郑州轻工业大学计算机与通信工程学院）；Kennesaw State University（肯尼索州立大学）；Michigan Technological University（密歇根理工大学）
MGML: A Plug-and-Play Meta-Guided Multi-Modal Learning Framework for Incomplete Multimodal Brain Tumor Segmentation
Nanchang University（南昌大学）；Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University（中山大学孙逸仙纪念医院）；Zhujiang Hospital, Southern Medical University（南方医科大学珠江医院）；University of California Los Angeles（加州大学洛杉矶分校）；Hubei University of Economics（湖北经济学院）；Xi'an Jiaotong Liverpool University（西交利物浦大学）；Southeast University（东南大学）
OmniBrainBench: A Comprehensive Multimodal Benchmark for Brain Imaging Analysis Across Multi-stage Clinical Tasks
Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen Memorial Hospital, Sun Yat-sen University（中山大学孙逸仙纪念医院）；Southern Medical University（南方医科大学）；Zhongshan Hospital, Fudan University（复旦大学中山医院）；Department of Neurosurgery, Prince of Wales Hospital（威尔士王室医院神经外科部）
Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation
ShanghaiTech University（上海科技大学）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）；Affiliated Hangzhou First People’s Hospital, Xihu University School of Medicine（西湖大学医学院附属杭州市第一人民医院）
