Accurate detection of anatomical landmarks is an essential step in several medical imaging tasks. We propose a novel communicative multi-agent reinforcement learning (C-MARL) system to automatically detect landmarks in 3D brain images. C-MARL enables the agents to learn explicit communication channels, as well as implicit communication signals by sharing certain weights of the architecture among all the agents. The proposed approach is evaluated on two brain imaging datasets from adult magnetic resonance imaging (MRI) and fetal ultrasound scans. Our experiments show that involving multiple cooperating agents by learning their communication with each other outperforms previous approaches using single agents.
The Chinese University of Hong Kong（香港中文大学）；Massachusetts Institute of Technology（麻省理工学院）
In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hin…
CABLD: Contrast-Agnostic Brain Landmark Detection with Consistency-Based Regularization
Department of Computer Science and Software Engineering, Concordia University（计算机科学与软件工程系，康科迪亚大学）；Department of Electrical and Computer Engineering, Concordia University（电气与计算机工程系，康科迪亚大学）
Automated Fetal Brain MRI Biometry in Healthy and Pathological Cases
Faculty of Medicine, University of Ljubljana（卢布尔雅那大学医学院）；University Medical Centre Ljubljana（卢布尔雅那大学医学中心）；Faculty of Electrical Engineering, University of Ljubljana（卢布尔雅那大学电气工程学院）
From Pre- to Intra-operative MRI: Predicting Brain Shift in Temporal Lobe Resection for Epilepsy Surgery
Surgical and Interventional Engineering（外科与介入工程）；King’s College London（伦敦国王学院）；National Hospital for Neurology and Neurosurgery（神经外科医院）；NHS（国家健康服务）；Neurosurgery, King’s College Hospital（神经外科，伦敦国王学院医院）；Fondazione IRCCS Ca’ Granda（IRCCS Ca’ Granda基金会）；Ospedale Maggiore di Milano（米兰大医院）
UCSD（加州大学圣地亚哥分校）；Salk Institute for Biological Studies（索尔克生物研究所）
Brightfield and fluorescent imaging of whole brain sections are funda- mental tools of research in mouse brain study. As sectioning and imaging become more efficient, there is an increasing need to automate the post-processing of sec- tions for alignment and three dimensional visualization. There is a further need to facilitate the development of a digital atlas, i.e. a brain-wide map annotated with cell type and tract tracing data, which would allow the automatic registra- tion of images stacks to a common coordinate system. Currently, registration of slices requires manual identification of …
We study a class of mathematical and statistical algorithms with the aim of establishing a computer-based framework for fast and reliable automatic abnormality detection on landmark represented image templates. Under this framework, we apply a landmark-based algorithm for finding a group average as an estimator that is said to best represent the common features of the group in study. This algorithm extracts information of momentum at each landmark through the process of template matching. If ever converges, the proposed algorithm produces a local coordinate system for each member of the observ…
The University of Tokyo（东京大学）；Computer Science and Artificial Intelligence Laboratory, MIT（麻省理工学院计算机科学与人工智能实验室）；Brigham and Women’s Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）；Ecole de Technologie Superieure（高等技术学院）；Lehigh University（里海大学）；RIKEN（理化学研究所）
A reliable Ultrasound (US)-to-US registration method to compensate for brain shift would substantially improve Image-Guided Neurological Surgery. Developing such a registration method is very challenging, due to factors such as missing correspondence in images, the complexity of brain pathology and the demand for fast computation. We propose a novel feature-driven active framework. Here, landmarks and their displacement are first estimated from a pair of US images using corresponding local image features. Subsequently, a Gaussian Process (GP) model is used to interpolate a dense deformation fi…
University College London（伦敦大学学院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）；Allen Institute for Brain Science（艾伦脑科学研究所）；Massachusetts Institute of Technology（麻省理工学院）
Nonlinear registration of 2D histological sections with corresponding slices of MRI data is a critical step of 3D histology reconstruction. This task is difficult due to the large differences in image contrast and resolution, as well as the complex nonrigid distortions produced when sectioning the sample and mounting it on the glass slide. It has been shown in brain MRI registration that better spatial alignment across modalities can be obtained by synthesizing one modality from the other and then using intra-modality registration metrics, rather than by using mutual information (MI) as metric…
Automatic Landmark-Based Segmentation of Human Subcortical Structures in MRI
École de technologie supérieure (ÉTS)（埃克塞尔技术高等学院）；Boston University School of Medicine（波士顿大学医学学院）；Signal and Smart Systems Lab (L3S)（信号与智能系统实验室）；National School of Engineering of Tunis, University of Tunis El Manar（突尼斯国家工程学院，突尼斯El Manar大学）
Deformable image registration is a key task in medical image analysis. The Brain Tumor Sequence Registration challenge (BraTS-Reg) aims at establishing correspondences between pre-operative and follow-up scans of the same patient diagnosed with an adult brain diffuse high-grade glioma and intends to address the challenging task of registering longitudinal data with major tissue appearance changes. In this work, we proposed a two-stage cascaded network based on the Inception and TransMorph models. The dataset for each patient was comprised of a native pre-contrast (T1), a contrast-enhanced T1-w…
The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients
Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine（印第安纳大学医学院计算病理学系）；Department of Electrical and Systems Engineering, Washington University in St. Louis（圣路易斯华盛顿大学电气与系统工程系）；GE HealthCare（GE医疗）；Center for Artificial Intelligence and Data Science for Integrated Diagnostics (AI2D) and Center for Biomedical Image Computing and Analytics (CBICA), University of Penn（宾夕法尼亚大学综合诊断人工智能与数据科学中心及生物医学图像计算与分析中心）
What neurosurgeons need to see: synthetic intra-operative MRI from ultrasound for brain-shift compensation in brain tumour surgery
Department of Neurosurgery, Neurovascular Unit, Río Hortega University Hospital, Valladolid, Spain（西班牙巴利亚多利德里奥·奥尔特加大学医院神经外科神经血管科）；Specialized Group in Biomedical Imaging and Computational Analysis (GEIBAC), Instituto de Investigación Biosanitaria de Valladolid (IBioVALL), Valladolid, Spain（西班牙巴利亚多利德生物医学研究与计算分析专业组(GEIBAC)，巴利亚多利德生物健康研究所(IBioVALL)）
Asynchronous Decentralized Federated Lifelong Learning for Landmark Localization in Medical Imaging
Rice University（莱斯大学）；McGovern Medical School, UTHealth（德克萨斯大学健康科学中心麦戈文医学院）；The University of Maryland Medical Intelligent Imaging (UM2ii) Center（马里兰大学医学智能成像中心）
Selective experience replay compression using coresets for lifelong deep reinforcement learning in medical imaging
Rice University（莱斯大学）；McGovern Medical School, UTHealth Houston（德克萨斯大学健康科学中心休斯顿麦戈文医学院）；The Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；University of Maryland School of Medicine（马里兰大学医学院）
Multi-environment lifelong deep reinforcement learning for medical imaging
Rice University（莱斯大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）；McGovern Medical School, UTHealth Houston（休斯顿UTHealth麦戈文医学院）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；University of Maryland Medical Intelligent Imaging (UM2ii)（马里兰大学医学智能成像中心）；University of Maryland School of Medicine（马里兰大学医学院）
Enhancing Alzheimer's Diagnosis: Leveraging Anatomical Landmarks in Graph Convolutional Neural Networks on Tetrahedral Meshes
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA（计算与增强智能学院，亚利桑那州立大学）；Amazon AGI, Redmond, WA, USA（亚马逊人工智慧部门）；Banner Alzheimer’s Institute, Phoenix, AZ, USA（贝纳尔德阿尔茨海默病研究所）
IIT Kharagpur（印度理工学院克勒格布尔分校）；IIT Bombay（印度理工学院孟买分校）；IIT Madras（印度理工学院马德拉斯分校）；Philips Research Philips Healthcare（飞利浦研究 飞利浦医疗保健）
In the medical field, landmark detection in MRI plays an important role in reducing medical technician efforts in tasks like scan planning, image registration, etc. First, 88 landmarks spread across the brain anatomy in the three respective views -- sagittal, coronal, and axial are manually annotated, later guidelines from the expert clinical technicians are taken sub-anatomy-wise, for better localization of the existing landmarks, in order to identify and locate the important atlas landmarks even in oblique scans. To overcome limited data availability, we implement realistic data augmentation…
The Brain Resection Multimodal Image Registration (ReMIND2Reg) 2025 Challenge
Inria Saclay Île-de-France（法国里昂萨克利研究所）；CEA（法国原子能委员会）；Université Paris-Saclay（巴黎-萨克利大学）；Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎脑研究所）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化技术研究所）；Inserm（法国国家医学研究院）；AP-HP（法国国家医院集团）；Hôpital de la Pitié Salpêtrière（皮蒂埃-萨尔普里埃尔医院）；Harvard Medical School（哈佛医学院）；Brigham and Women's Hospital（布里根医院）；Department of Radiology and Radiological Science（放射学与放射科学系）；Johns Hopkins Medical School（约翰霍普金斯医学院）；Institute of Medical Informatics（医学信息学研究所）；University of Lübeck（吕贝克大学）；Image Analysis and Communications Laboratory（图像分析与通信实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）；Johns Hopkins University（约翰霍普金斯大学）；CSAIL（媒体实验室）
Community-Level Modeling of Gyral Folding Patterns for Robust and Anatomically Informed Individualized Brain Mapping
Department of Computer Science and Engineering, University of Texas at Arlington（计算机科学与工程系，德克萨斯大学阿灵顿分校）；School of Computing, University of Georgia（计算学院，佐治亚大学）；Department of Computer Science, Indiana University Indianapolis（计算机科学系，印第安纳大学印第安纳波利斯分校）
Univ Lyon（里昂大学）；INSA‐Lyon（里昂国立应用科学学院）；Université Claude Bernard Lyon 1（里昂第一大学）；UJM-Saint Etienne（圣艾蒂安大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；CREATIS UMR 5220（CREATIS联合研究单位5220）；U1206（U1206研究单位）
Semantic segmentation is an established while rapidly evolving field in medical imaging. In this paper we focus on the segmentation of brain Magnetic Resonance Images (MRI) into cerebral structures using convolutional neural networks (CNN). CNNs achieve good performance by finding effective high dimensional image features describing the patch content only. In this work, we propose different ways to introduce spatial constraints into the network to further reduce prediction inconsistencies. A patch based CNN architecture was trained, making use of multiple scales to gather contextual informatio…
A benchmark for 2D foetal brain ultrasound analysis
Brain and Mind Centre, University of Sydney（悉尼大学脑与思维中心）；Faculty Of Science, Yamagata University（山形大学理学院）；Hospital Universitari de Girona Doctor Josep Trueta（赫罗纳大学医院何塞普·特鲁埃塔医生医院）
Queen Elizabeth Hospital（伊丽莎白女王医院）；The Chinese University of Hong Kong（香港中文大学）
Brain midline shift (MLS) is one of the most critical factors to be considered for clinical diagnosis and treatment decision-making for intracranial hemorrhage. Existing computational methods on MLS quantification not only require intensive labeling in millimeter-level measurement but also suffer from poor performance due to their dependence on specific landmarks or simplified anatomical assumptions. In this paper, we propose a novel semi-supervised framework to accurately measure the scale of MLS from head CT scans. We formulate the MLS measurement task as a deformation estimation problem and…
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Trinity College Dublin（都柏林圣三一学院）
Intra-operative ultrasound is an increasingly important imaging modality in neurosurgery. However, manual interaction with imaging data during the procedures, for example to select landmarks or perform segmentation, is difficult and can be time consuming. Yet, as registration to other imaging modalities is required in most cases, some annotation is necessary. We propose a segmentation method based on DeepVNet and specifically evaluate the integration of pre-training with simulated ultrasound sweeps to improve automatic segmentation and enable a fully automatic initialization of registration. I…
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）；Rutgers, The State University of New Jersey（罗格斯新泽西州立大学）
Automatic segmentation of anatomical landmarks from ultrasound (US) plays an important role in the management of preterm neonates with a very low birth weight due to the increased risk of developing intraventricular hemorrhage (IVH) or other complications. One major problem in developing an automatic segmentation method for this task is the limited availability of annotated data. To tackle this issue, we propose a novel image synthesis method using multi-scale self attention generator to synthesize US images from various segmentation masks. We show that our method can synthesize high-quality U…
Origin Health India（印度Origin Health公司）；Bangalore Fetal Medicine Center（班加罗尔胎儿医学中心）；Prime Imaging and Prenatal Diagnostics（Prime影像与产前诊断中心）；Kasturba Medical College（卡斯图尔巴医学院）；Manipal Academy of Higher Education（马尼帕尔高等教育学院）；Origin Health（Origin Health公司）
Multiple studies have demonstrated that obtaining standardized fetal brain biometry from mid-trimester ultrasonography (USG) examination is key for the reliable assessment of fetal neurodevelopment and the screening of central nervous system (CNS) anomalies. Obtaining these measurements is highly subjective, expertise-driven, and requires years of training experience, limiting quality prenatal care for all pregnant mothers. In this study, we propose a deep learning (DL) approach to compute 3 key fetal brain biometry from the 2D USG images of the transcerebellar plane (TC) through the accurate …
Johns Hopkins University（约翰斯·霍普金斯大学）；Rutgers, The State University of New Jersey（罗格斯大学，新泽西州立大学）
Due to its excellent performance, U-Net is the most widely used backbone architecture for biomedical image segmentation in the recent years. However, in our studies, we observe that there is a considerable performance drop in the case of detecting smaller anatomical landmarks with blurred noisy boundaries. We analyze this issue in detail, and address it by proposing an over-complete architecture (Ki-Net) which involves projecting the data onto higher dimensions (in the spatial sense). This network, when augmented with U-Net, results in significant improvements in the case of segmenting small a…
We propose a fully automatic method to find standardized view planes in 3D image acquisitions. Standard view images are important in clinical practice as they provide a means to perform biometric measurements from similar anatomical regions. These views are often constrained to the native orientation of a 3D image acquisition. Navigating through target anatomy to find the required view plane is tedious and operator-dependent. For this task, we employ a multi-scale reinforcement learning (RL) agent framework and extensively evaluate several Deep Q-Network (DQN) based strategies. RL enables a na…
UCL Queen Square Institute of Neurology（伦敦大学学院皇后广场神经学研究所）
This paper presents a framework for automatically learning shape and appearance models for medical (and certain other) images. It is based on the idea that having a more accurate shape and appearance model leads to more accurate image registration, which in turn leads to a more accurate shape and appearance model. This leads naturally to an iterative scheme, which is based on a probabilistic generative model that is fit using Gauss-Newton updates within an EM-like framework. It was developed with the aim of enabling distributed privacy-preserving analysis of brain image data, such that shared …
