Automatic pancreas segmentation in radiology images, eg., computed tomography (CT) and magnetic resonance imaging (MRI), is frequently required by computer-aided screening, diagnosis, and quantitative assessment. Yet pancreas is a challenging abdominal organ to segment due to the high inter-patient anatomical variability in both shape and volume metrics. Recently, convolutional neural networks (CNNs) have demonstrated promising performance on accurate segmentation of pancreas. However, the CNN-based method often suffers from segmentation discontinuity for reasons such as noisy image quality and blurry pancreatic boundary. From this point, we propose to introduce recurrent neural networks (RNNs) to address the problem of spatial non-smoothness of inter-slice pancreas segmentation across adjacent image slices. To inference initial segmentation, we first train a 2D CNN sub-network, where we modify its network architecture with deep-supervision and multi-scale feature map aggregation so that it can be trained from scratch with small-sized training data and presents superior performance than transferred models. Thereafter, the successive CNN outputs are processed by another RNN sub-network, which refines the consistency of segmented shapes. More specifically, the RNN sub-network consists convolutional long short-term memory (CLSTM) units in both top-down and bottom-up directions, which regularizes the segmentation of an image by integrating predictions of its neighboring slices. We train the stacked CNN-RNN model end-to-end and perform quantitative evaluations on both CT and MRI images.
University of Florida（佛罗里达大学）；National Institutes of Health Clinical Center（国立卫生研究院临床中心）
Deep neural networks have demonstrated very promising performance on accurate segmentation of challenging organs (e.g., pancreas) in abdominal CT and MRI scans. The current deep learning approaches conduct pancreas segmentation by processing sequences of 2D image slices independently through deep, dense per-pixel masking for each image, without explicitly enforcing spatial consistency constraint on segmentation of successive slices. We propose a new convolutional/recurrent neural network architecture to address the contextual learning and segmentation consistency problem. A deep convolutional …
A Multi-Stage Fine-Tuning and Ensembling Strategy for Pancreatic Tumor Segmentation in Diagnostic and Therapeutic MRI
German Cancer Research Center (DKFZ) Heidelberg（德国癌症研究中心（DKFZ）海德堡）；Faculty of Mathematics and Computer Science（数学与计算机科学学院）；Heidelberg University（海德堡大学）；Helmholtz Imaging, DKFZ（海德堡影像中心，DKFZ）；Pattern Analysis and Learning Group（模式分析与学习小组）；Department of Radiation Oncology（放射肿瘤科）；Heidelberg University Hospital（海德堡大学医院）；International School of Medicine（国际医学院）；Istanbul Medipol University（伊斯坦布尔梅迪波尔大学）
Pediatric Pancreas Segmentation from MRI Scans with Deep Learning
Northwestern University（西北大学）；Gazi University Faculty of Medicine（加齐大学医学院）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Washington University School of Medicine in St. Louis（圣路易斯华盛顿大学医学院）；University of Iowa, Carver College of Medicine（爱荷华大学卡弗医学院）
Adaptive Aggregation Weights for Federated Segmentation of Pancreas MRI
Northwestern University（西北大学）；Istanbul University（伊斯坦布尔大学）；University of Catania（卡塔尼亚大学）；Erasmus Medical Center（伊拉斯姆斯医学中心）；University of Wisconsin（威斯康星大学）；Mayo Clinic（梅奥诊所）；New York University（纽约大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；NVIDIA（英伟达公司）
Cyst-X: A Multi-Center MRI Benchmark and Federated Learning Framework for Malignancy-Risk Stratification of Pancreatic Cystic Neoplasm
Machine & Hybrid Intelligence Lab, Department of Radiology, Northwestern University（机器与混合智能实验室，放射科，西北大学）；Istanbul Faculty of Medicine, Istanbul University（伊斯坦布尔大学医学学院）；Department of Biomedical Engineering and Radiology, University of Wisconsin-Madison（生物医学工程与放射科，威斯康星大学麦迪逊分校）；Department of Preventive Medicine, Northwestern University（预防医学系，西北大学）；Division of Gastroenterology and Hepatology, New York University（消化内科与肝病科，纽约大学）；Department of Electrical, Electronic and Computer Engineering, University of Catania（电气、电子和计算机工程系，卡塔尼亚大学）；NVIDIA；Department of Radiology, Columbia University（放射科，哥伦比亚大学）；Department of Radiology and Nuclear Medicine, Erasmus Medical Center（放射科与核医学科，埃因霍温医学院）；Department of Gastroenterology and Hepatology, Erasmus Medical Center（消化内科与肝病科，埃因霍温医学院）；Department of Radiology, New York University（放射科，纽约大学）；Division of Gastroenterology and Hepatology, Mayo Clinic Florida（消化内科与肝病科，迈阿密诊所佛罗里达分部）；Department of Gastroenterology and Hepatology, Northwestern University（消化内科与肝病科，西北大学）
Segmentation of pancreas is important for medical image analysis, yet it faces great challenges of class imbalance, background distractions and non-rigid geometrical features. To address these difficulties, we introduce a Deep Q Network(DQN) driven approach with deformable U-Net to accurately segment the pancreas by explicitly interacting with contextual information and extract anisotropic features from pancreas. The DQN based model learns a context-adaptive localization policy to produce a visually tightened and precise localization bounding box of the pancreas. Furthermore, deformable U-Net …
University of Catania（卡塔尼亚大学）；Northwestern University（西北大学）；Mayo Clinic（梅奥诊所）；University of Torino（都灵大学）
Early detection of precancerous cysts or neoplasms, i.e., Intraductal Papillary Mucosal Neoplasms (IPMN), in pancreas is a challenging and complex task, and it may lead to a more favourable outcome. Once detected, grading IPMNs accurately is also necessary, since low-risk IPMNs can be under surveillance program, while high-risk IPMNs have to be surgically resected before they turn into cancer. Current standards (Fukuoka and others) for IPMN classification show significant intra- and inter-operator variability, beside being error-prone, making a proper diagnosis unreliable. The established prog…
CrossPan: A Comprehensive Benchmark for Cross-Sequence Pancreas MRI Segmentation and Generalization
Department of Radiology, Northwestern University（西北大学放射科）；Department of Electrical and Computer Engineering, Northwestern University（西北大学电气与计算机工程系）；Department of Computer Science, Northwestern University（西北大学计算机科学系）；Department of Gastroenterology, New York University（纽约大学消化内科部）；Division of Gastroenterology and Hepatology, Mayo Clinic in Florida（佛罗里达梅奥诊所消化内科与肝病学部）；Department of Computer Science, Johns Hopkins University（约翰霍普金斯大学计算机科学系）；Departments of Gastroenterology and Hepatology, Northwestern University（西北大学消化内科与肝病学部）
PAII Inc.（PAII公司）；Shengjing Hospital of China Medical University（中国医科大学附属盛京医院）；Changhai Hospital（长海医院）；Ping An Technology Co., Ltd.（平安科技（深圳）有限公司）
Accurate and automated tumor segmentation is highly desired since it has the great potential to increase the efficiency and reproducibility of computing more complete tumor measurements and imaging biomarkers, comparing to (often partial) human measurements. This is probably the only viable means to enable the large-scale clinical oncology patient studies that utilize medical imaging. Deep learning approaches have shown robust segmentation performances for certain types of tumors, e.g., brain tumors in MRI imaging, when a training dataset with plenty of pixel-level fully-annotated tumor images…
Brain Tumor Segmentation in MRI Images with 3D U-Net and Contextual Transformer
University of Information Technology（信息科技大学）；Vietnam National University（越南国家大学）；College of Technology and Design, University of Economics Ho Chi Minh City（胡志明市经济大学技术与设计学院）；Ho Chi Minh City Open University（胡志明市开放大学）
Accurate and robust segmentation of small organs in whole-body MRI is difficult due to anatomical variation and class imbalance. Recent deep network based approaches have demonstrated promising performance on abdominal multi-organ segmentations. However, the performance on small organs is still suboptimal as these occupy only small regions of the whole-body volumes with unclear boundaries and variable shapes. A coarse-to-fine, hierarchical strategy is a common approach to alleviate this problem, however, this might miss useful contextual information. We propose a two-stage approach with weight…
Influence of Early through Late Fusion on Pancreas Segmentation from Imperfectly Registered Multimodal MRI
Vanderbilt University（范德堡大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；National Institutes of Health（美国国立卫生研究院）；University of Texas at Austin（得克萨斯大学奥斯汀分校）；Vanderbilt University Medical Center（范德堡大学医学中心）；VA Tennessee Valley Healthcare System（退伍军人事务部田纳西河谷医疗系统）
University of Central Florida（中佛罗里达大学）；Foundation for Research and Technology Hellas（希腊研究与技术基金会）；Mayo Clinic（梅奥诊所）
Intraductal papillary mucinous neoplasm (IPMN) is a precursor to pancreatic ductal adenocarcinoma. While over half of patients are diagnosed with pancreatic cancer at a distant stage, patients who are diagnosed early enjoy a much higher 5-year survival rate of $34\%$ compared to $3\%$ in the former; hence, early diagnosis is key. Unique challenges in the medical imaging domain such as extremely limited annotated data sets and typically large 3D volumetric data have made it difficult for deep learning to secure a strong foothold. In this work, we construct two novel "inflated" deep network arch…
Artificial Intelligence (Deep Learning(DL)/ Machine Learning(ML)) techniques are widely being used to address and overcome all kinds of ill-posed problems in medical imaging which was or in fact is seemingly impossible. Reducing gradient directions but harnessing high angular resolution(HAR) diffusion data in MR that retains clinical features is an important and challenging problem in the field. While the DL/ML approaches are promising, it is important to incorporate relevant context for the data to ensure that maximum prior information is provided for the AI model to infer the posterior. In t…
Pancreas Part Segmentation under Federated Learning Paradigm
Northwestern University, Evanston, IL, USA（西北大学）；Indiana University–Purdue University Indianapolis, Indianapolis, IN, USA（印第安纳大学-普渡大学印第安纳波利斯分校）；University of Catania, Catania, Italy（卡塔尼亚大学）；Johns Hopkins University, Baltimore, MD, USA（约翰霍普金斯大学）；New York University Langone Health, NY, USA（纽约大学Langone健康中心）；Mayo Clinic, Rochester, MN, USA（梅奥诊所）；University of Illinois Chicago, Chicago, IL, USA（伊利诺伊大学芝加哥分校）；Istanbul University, Istanbul, Turkey（伊斯坦布尔大学）
Numerous factors could lead to partial deteriorations of medical images. For example, metallic implants will lead to localized perturbations in MRI scans. This will affect further post-processing tasks such as attenuation correction in PET/MRI or radiation therapy planning. In this work, we propose the inpainting of medical images via Generative Adversarial Networks (GANs). The proposed framework incorporates two patch-based discriminator networks with additional style and perceptual losses for the inpainting of missing information in realistically detailed and contextually consistent manner. …
BrainFusionNet: a deep learning and XAI model to understand local, global, and sequential features of MRI images for improved brain tumour detection
School of Mathematics, Physics and Computing, University of Southern Queensland（南方昆士兰大学数学、物理与计算学院）；School of Engineering, University of Southern Queensland（南方昆士兰大学工程学院）
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）
This paper reports Deep LOGISMOS approach to 3D tumor segmentation by incorporating boundary information derived from deep contextual learning to LOGISMOS - layered optimal graph image segmentation of multiple objects and surfaces. Accurate and reliable tumor segmentation is essential to tumor growth analysis and treatment selection. A fully convolutional network (FCN), UNet, is first trained using three adjacent 2D patches centered at the tumor, providing contextual UNet segmentation and probability map for each 2D patch. The UNet segmentation is then refined by Gaussian Mixture Model (GMM) a…
DINOv3-Guided Cross Fusion Framework for Semantic-aware CT generation from MRI and CBCT
Faculty of Information Technology, Monash University（墨尔本大学信息科技学院）；Institute of Artificial Intelligence, School of Computer Science, Wuhan University（武汉大学计算机科学学院人工智能研究所）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室）
Zhejiang University（浙江大学）；University of Notre Dame（圣母大学）
Automatic segmentation of cardiac magnetic resonance imaging (MRI) facilitates efficient and accurate volume measurement in clinical applications. However, due to anisotropic resolution and ambiguous border (e.g., right ventricular endocardium), existing methods suffer from the degradation of accuracy and robustness in 3D cardiac MRI video segmentation. In this paper, we propose a novel Deformable U-Net (DeU-Net) to fully exploit spatio-temporal information from 3D cardiac MRI video, including a Temporal Deformable Aggregation Module (TDAM) and a Deformable Global Position Attention (DGPA) net…
The rapid increment of morbidity of brain stroke in the last few years have been a driving force towards fast and accurate segmentation of stroke lesions from brain MRI images. With the recent development of deep-learning, computer-aided and segmentation methods of ischemic stroke lesions have been useful for clinicians in early diagnosis and treatment planning. However, most of these methods suffer from inaccurate and unreliable segmentation results because of their inability to capture sufficient contextual features from the MRI volumes. To meet these requirements, 3D convolutional neural ne…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
Vision transformers, with their ability to more efficiently model long-range context, have demonstrated impressive accuracy gains in several computer vision and medical image analysis tasks including segmentation. However, such methods need large labeled datasets for training, which is hard to obtain for medical image analysis. Self-supervised learning (SSL) has demonstrated success in medical image segmentation using convolutional networks. In this work, we developed a \underline{s}elf-distillation learning with \underline{m}asked \underline{i}mage modeling method to perform SSL for vision \u…
Revolutionizing Glioma Segmentation & Grading Using 3D MRI - Guided Hybrid Deep Learning Models
Department of Computer Science and Engineering, School of Computer Science and Engineering（计算机科学与工程系，计算机科学与工程学院）；Vellore Institute of Technology, Chennai, India（维洛雷技术学院，钦奈，印度）；School of Advanced Science, Vellore Institute of Technology, Chennai, India（先进科学学院，维洛雷技术学院，钦奈，印度）；Centre for Human Movement Analytics, Vellore Institute of Technology, Chennai, India（人类运动分析中心，维洛雷技术学院，钦奈，印度）
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for shor…
University of Arkansas（阿肯色大学）；Concordia University（康考迪亚大学）；Carnegie Mellon University（卡内基梅隆大学）
In recent years, deep neural networks have achieved state-of-the-art performance in a variety of recognition and segmentation tasks in medical imaging including brain tumor segmentation. We investigate that segmenting a brain tumor is facing to the imbalanced data problem where the number of pixels belonging to the background class (non tumor pixel) is much larger than the number of pixels belonging to the foreground class (tumor pixel). To address this problem, we propose a multi-task network which is formed as a cascaded structure. Our model consists of two targets, i.e., (i) effectively dif…
The University of Iowa（爱荷华大学）；University of Iowa Carver College of Medicine（爱荷华大学卡佛医学院）
A fully automated knee MRI segmentation method to study osteoarthritis (OA) was developed using a novel hierarchical set of random forests (RF) classifiers to learn the appearance of cartilage regions and their boundaries. A neighborhood approximation forest is used first to provide contextual feature to the second-level RF classifier that also considers local features and produces location-specific costs for the layered optimal graph image segmentation of multiple objects and surfaces (LOGISMOS) framework. Double echo steady state (DESS) MRIs used in this work originated from the Osteoarthrit…
Technical University of Berlin, Germany（柏林工业大学（德国））；Hasso Plattner Institute for Digital Engineering, University of Potsdam, Germany（波茨坦大学哈索普拉特纳数字工程研究所（德国））；Hasso Plattner Institute for Digital Health at the Icahn School of Medicine at Mount Sinai, NYC, USA（西奈山伊坎医学院哈索普拉特纳数字健康研究所（美国纽约市））
Self-supervised learning methods can be used to learn meaningful representations from unlabeled data that can be transferred to supervised downstream tasks to reduce the need for labeled data. In this paper, we propose a 3D self-supervised method that is based on the contrastive (SimCLR) method. Additionally, we show that employing Bayesian neural networks (with Monte-Carlo Dropout) during the inference phase can further enhance the results on the downstream tasks. We showcase our models on two medical imaging segmentation tasks: i) Brain Tumor Segmentation from 3D MRI, ii) Pancreas Tumor Segm…
University of Brescia（布雷西亚大学）；University of Glasgow（格拉斯哥大学）
Many functional and structural neuroimaging studies call for accurate morphometric segmentation of different brain structures starting from image intensity values of MRI scans. Current automatic (multi-) atlas-based segmentation strategies often lack accuracy on difficult-to-segment brain structures and, since these methods rely on atlas-to-scan alignment, they may take long processing times. Recently, methods deploying solutions based on Convolutional Neural Networks (CNNs) are making the direct analysis of out-of-the-scanner data feasible. However, current CNN-based solutions partition the t…
Bilkent University（比尔肯特大学）；National Magnetic Resonance Research Center (UMRAM)（国家磁共振研究中心（UMRAM））；Sabuncu Brain Research Center（萨本库脑研究中心）
Generative adversarial models with convolutional neural network (CNN) backbones have recently been established as state-of-the-art in numerous medical image synthesis tasks. However, CNNs are designed to perform local processing with compact filters, and this inductive bias compromises learning of contextual features. Here, we propose a novel generative adversarial approach for medical image synthesis, ResViT, that leverages the contextual sensitivity of vision transformers along with the precision of convolution operators and realism of adversarial learning.} ResViT's generator employs a cent…
Active learning is a unique abstraction of machine learning techniques where the model/algorithm could guide users for annotation of a set of data points that would be beneficial to the model, unlike passive machine learning. The primary advantage being that active learning frameworks select data points that can accelerate the learning process of a model and can reduce the amount of data needed to achieve full accuracy as compared to a model trained on a randomly acquired data set. Multiple frameworks for active learning combined with deep learning have been proposed, and the majority of them …
SSFMamba: Learning Symmetry-driven Spatial-Frequency Modeling for Physically Consistent 3D Medical Image Segmentation
State Key Laboratory of Networking and Switching Technology, School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，计算机学院（国家试点软件工程学院），北京邮电大学）；School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（智能工程与自动化学院，北京邮电大学）；School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）
Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation
Université de technologie de Compiègne（贡比涅技术大学）；CNRS（法国国家科学研究中心）；Université de Rouen Normandie（鲁昂诺曼底大学）；Centre Henri Becquerel（亨利·贝克勒尔中心）；Institut universitaire de France（法国大学研究院）
On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstruction
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（麻省总医院Athinoula A. Martinos生物医学成像中心）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）
I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling
Department of Electrical and Electronics Engineering, and National Magnetic Resonance Research Center, Bilkent University（电子工程系和国家磁共振研究中心，比尔肯特大学）；Department of Electrical Engineering, Stanford University（电气工程系，斯坦福大学）
Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam Cardiovascular Sciences（阿姆斯特丹心血管科学研究所）；Informatics Institute, University of Amsterdam（阿姆斯特丹大学信息学研究所）
High-resolution medical images are beneficial for analysis but their acquisition may not always be feasible. Alternatively, high-resolution images can be created from low-resolution acquisitions using conventional upsampling methods, but such methods cannot exploit high-level contextual information contained in the images. Recently, better performing deep-learning based super-resolution methods have been introduced. However, these methods are limited by their supervised character, i.e. they require high-resolution examples for training. Instead, we propose an unsupervised deep learning semanti…
