Universal lesion detection from computed tomography (CT) slices is important for comprehensive disease screening. Since each lesion can locate in multiple adjacent slices, 3D context modeling is of great significance for developing automated lesion detection algorithms. In this work, we propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) that leverages depthwise separable convolutional filters and a group transform module (GTM) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. To facilitate faster convergence, a novel 3D network pre-training method is derived using solely large-scale 2D object detection dataset in the natural image domain. We demonstrate that with the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset (3.48% absolute improvement in the sensitivity of FPs@0.5), significantly surpassing the baseline method by up to 6.06% (in MAP@0.5) which adopts 2D convolution for 3D context modeling. Moreover, the proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
National Institutes of Health（美国国立卫生研究院）；University of Florida（佛罗里达大学）；Nvidia Corp（英伟达公司）；Ping An Insurance (Group) Company of China Ltd（中国平安保险（集团）股份有限公司）
Volumetric lesion segmentation via medical imaging is a powerful means to precisely assess multiple time-point lesion/tumor changes. Because manual 3D segmentation is prohibitively time consuming and requires radiological experience, current practices rely on an imprecise surrogate called response evaluation criteria in solid tumors (RECIST). Despite their coarseness, RECIST marks are commonly found in current hospital picture and archiving systems (PACS), meaning they can provide a potentially powerful, yet extraordinarily challenging, source of weak supervision for full 3D segmentation. Towa…
The University of Sydney（悉尼大学）；Sydney Neuroimaging Analysis Centre（悉尼神经影像分析中心）；NVIDIA Corporation（英伟达公司）
Federated learning (FL) has been widely employed for medical image analysis to facilitate multi-client collaborative learning without sharing raw data. Despite great success, FL's performance is limited for multiple sclerosis (MS) lesion segmentation tasks, due to variance in lesion characteristics imparted by different scanners and acquisition parameters. In this work, we propose the first FL MS lesion segmentation framework via two effective re-weighting mechanisms. Specifically, a learnable weight is assigned to each local node during the aggregation process, based on its segmentation perfo…
Accurate segmentation of breast lesions is a crucial step in evaluating the characteristics of tumors. However, this is a challenging task, since breast lesions have sophisticated shape, topological structure, and variation in the intensity distribution. In this paper, we evaluated the performance of three unsupervised algorithms for the task of breast Magnetic Resonance (MRI) lesion segmentation, namely, Gaussian Mixture Model clustering, K-means clustering and a marker-controlled Watershed transformation based method. All methods were applied on breast MRI slices following selection of regio…
Lesion segmentation in medical imaging serves as an effective tool for assessing tumor sizes and monitoring changes in growth. However, not only is manual lesion segmentation time-consuming, but it is also expensive and requires expert radiologist knowledge. Therefore many hospitals rely on a loose substitute called response evaluation criteria in solid tumors (RECIST). Although these annotations are far from precise, they are widely used throughout hospitals and are found in their picture archiving and communication systems (PACS). Therefore, these annotations have the potential to serve as a…
Incorporating data-specific domain knowledge in deep networks explicitly can provide important cues beneficial for lesion detection and can mitigate the need for diverse heterogeneous datasets for learning robust detectors. In this paper, we exploit the domain information present in computed tomography (CT) scans and propose a robust universal lesion detection (ULD) network that can detect lesions across all organs of the body by training on a single dataset, DeepLesion. We analyze CT-slices of varying intensities, generated using heuristically determined Hounsfield Unit(HU) windows that indiv…
Transfer learning and joint learning approaches are extensively used to improve the performance of Convolutional Neural Networks (CNNs). In medical imaging applications in which the target dataset is typically very small, transfer learning improves feature learning while joint learning has shown effectiveness in improving the network's generalization and robustness. In this work, we study the combination of these two approaches for the problem of liver lesion segmentation and classification. For this purpose, 332 abdominal CT slices containing lesion segmentation and classification of three le…
Modeling 3D context is essential for high-performance 3D medical image analysis. Although 2D networks benefit from large-scale 2D supervised pretraining, it is weak in capturing 3D context. 3D networks are strong in 3D context yet lack supervised pretraining. As an emerging technique, \emph{3D context fusion operator}, which enables conversion from 2D pretrained networks, leverages the advantages of both and has achieved great success. Existing 3D context fusion operators are designed to be spatially symmetric, i.e., performing identical operations on each 2D slice like convolutions. However, …
Shanghai Jiao Tong University（上海交通大学）；Dianei Technology（点内科技）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（人工智能教育部重点实验室，人工智能研究院，上海交通大学）；Shanghai Chest Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属上海胸科医院）；Huawei Hisilicon（华为海思）
This paper addresses a fundamental challenge in 3D medical image processing: how to deal with imaging thickness. For anisotropic medical volumes, there is a significant performance gap between thin-slice (mostly 1mm) and thick-slice (mostly 5mm) volumes. Prior arts tend to use 3D approaches for the thin-slice and 2D approaches for the thick-slice, respectively. We aim at a unified approach for both thin- and thick-slice medical volumes. Inspired by recent advances in video analysis, we propose AlignShift, a novel parameter-free operator to convert theoretically any 2D pretrained network into t…
Dual-Encoder Transformer-Based Multimodal Learning for Ischemic Stroke Lesion Segmentation Using Diffusion MRI
Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University（麻醉学、围术期与疼痛医学系，斯坦福大学）；Department of Biomedical Sciences, Seoul National University（生物医学科学系，首尔国立大学）；Department of Computer Engineering, National University of Sciences and Technology (NUST)（计算机工程系，国立科学与技术大学（NUST））；Department of Computer Science, The University of Alabama（计算机科学系，阿拉巴马大学）；Department of Computer Science, Hanyang University（计算机科学系，翰阳大学）
Faculty of Information Technology, University of Science, VNU-HCM（越南胡志明市国家大学科学大学信息技术学院）；SBEI School №1228 ”Lefortovo”（第1228号“列福尔托沃”基础普通教育学校）
In the development of technology, there are increasing cases of brain disease, there are more treatments proposed and achieved a positive result. However, with Brain-Lesion, the early diagnoses can improve the possibility for successful treatment and can help patients recuperate better. From this reason, Brain-Lesion is one of the controversial topics in medical images analysis nowadays. With the improvement of the architecture, there is a variety of methods that are proposed and achieve competitive scores. In this paper, we proposed a technique that uses efficient-net for 3D images, especiall…
National Institutes of Health（美国国立卫生研究院）；University of Florida（佛罗里达大学）；Ping An Insurance (Group) Company of China, Ltd.（中国平安保险（集团）股份有限公司）
Volumetric lesion segmentation from computed tomography (CT) images is a powerful means to precisely assess multiple time-point lesion/tumor changes. However, because manual 3D segmentation is prohibitively time consuming, current practices rely on an imprecise surrogate called response evaluation criteria in solid tumors (RECIST). Despite their coarseness, RECIST markers are commonly found in current hospital picture and archiving systems (PACS), meaning they can provide a potentially powerful, yet extraordinarily challenging, source of weak supervision for full 3D segmentation. Toward this e…
New Jersey Institute of Technology（新泽西理工学院）；Kessler Foundation（凯斯勒基金会）；Rutgers – New Jersey Medical School（罗格斯大学新泽西医学院）；Medical College of Wisconsin（威斯康星医学院）；Rutgers University – Newark（罗格斯大学纽瓦克分校）
Automatic identification of brain lesions from magnetic resonance imaging (MRI) scans of stroke survivors would be a useful aid in patient diagnosis and treatment planning. We propose a multi-modal multi-path convolutional neural network system for automating stroke lesion segmentation. Our system has nine end-to-end UNets that take as input 2-dimensional (2D) slices and examines all three planes with three different normalizations. Outputs from these nine total paths are concatenated into a 3D volume that is then passed to a 3D convolutional neural network to output a final lesion mask. We tr…
Revisiting Lesion Tracking in 3D Total Body Photography
Johns Hopkins University（约翰斯·霍普金斯大学）；Eunice Kennedy Shriver National Institute of Child Health and Human Development（尤妮斯·肯尼迪·施莱弗国家儿童健康与人类发展研究所）；Lumo Imaging（卢莫影像公司）；Johns Hopkins School of Medicine（约翰斯·霍普金斯医学院）；University of Arkansas（阿肯色大学）
Siemens Healthineers, X-ray products（西门子医疗 X 射线产品部）；Siemens Healthineers, Medical Imaging Technologies（西门子医疗医学影像技术部）；School of Information Technologies, University of Sydney（悉尼大学信息技术学院）
While deep convolutional neural networks (CNN) have been successfully applied for 2D image analysis, it is still challenging to apply them to 3D anisotropic volumes, especially when the within-slice resolution is much higher than the between-slice resolution and when the amount of 3D volumes is relatively small. On one hand, direct learning of CNN with 3D convolution kernels suffers from the lack of data and likely ends up with poor generalization; insufficient GPU memory limits the model size or representational power. On the other hand, applying 2D CNN with generalizable features to 2D slice…
Volumetric segmentation of lesions on CT scans is important for many types of analysis, including lesion growth kinetic modeling in clinical trials and machine learning of radiomic features. Manual segmentation is laborious, and impractical for large-scale use. For routine clinical use, and in clinical trials that apply the Response Evaluation Criteria In Solid Tumors (RECIST), clinicians typically outline the boundaries of a lesion on a single slice to extract diameter measurements. In this work, we have collected a large-scale database, named LesionVis, with pixel-wise manual 2D lesion delin…
School of Information Technology and Electrical Engineering, University of Queensland（昆士兰大学信息技术与电气工程学院）；Science and Engineering Faculty, Queensland University of Technology（昆士兰科技大学科学与工程学院）
Previous studies on computer aided detection/diagnosis (CAD) in 4D breast magnetic resonance imaging (MRI) regard lesion detection, segmentation and characterization as separate tasks, and typically require users to manually select 2D MRI slices or regions of interest as the input. In this work, we present a breast MRI CAD system that can handle 4D multimodal breast MRI data, and integrate lesion detection, segmentation and characterization with no user intervention. The proposed CAD system consists of three major stages: region candidate generation, feature extraction and region candidate cla…
NVIDIA AI Technology Center（英伟达人工智能技术中心）；Nanyang Technological University（南洋理工大学）；Monash University（莫纳什大学）
Lesion detection from computed tomography (CT) scans is challenging compared to natural object detection because of two major reasons: small lesion size and small inter-class variation. Firstly, the lesions usually only occupy a small region in the CT image. The feature of such small region may not be able to provide sufficient information due to its limited spatial feature resolution. Secondly, in CT scans, the lesions are often indistinguishable from the background since the lesion and non-lesion areas may have very similar appearances. To tackle both problems, we need to enrich the feature …
Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation in Medical Images: An Experimental Study
The Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心放射肿瘤学系医学人工智能与自动化（MAIA）实验室）；Medical Image Processing Group, Department of Radiology, University of Pennsylvania（宾夕法尼亚大学放射学系医学图像处理组）；Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心内科学系消化与肝脏疾病科）
State-of-the-Art Stroke Lesion Segmentation at 1/1000th of Parameters
Center for Data Science, Nell Hodgson Woodruff School of Nursing（内尔·霍奇森·伍德拉夫护理学院数据科学中心）；Emory University（埃默里大学）；University of South Carolina（南卡罗来纳大学）；Georgia State University（佐治亚州立大学）
Eastern Hepatobiliary Surgery Hospital（东方肝胆外科医院）；Ping An Technology（平安科技）；Georgia Institute of Technology（佐治亚理工学院）；PAII Inc.（PAII公司）
Objectives: to propose a fully-automatic computer-aided diagnosis (CAD) solution for liver lesion characterization, with uncertainty estimation. Methods: we enrolled 400 patients who had either liver resection or a biopsy and was diagnosed with either hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma, or secondary metastasis, from 2006 to 2019. Each patient was scanned with T1WI, T2WI, T1WI venous phase (T2WI-V), T1WI arterial phase (T1WI-A), and DWI MRI sequences. We propose a fully-automatic deep CAD pipeline that localizes lesions from 3D MRI studies using key-slice parsing an…
Heidelberg University（海德堡大学）；University Medical Center Mannheim（曼海姆大学医学中心）
At present, lesion segmentation is still performed manually (or semi-automatically) by medical experts. To facilitate this process, we contribute a fully-automatic lesion segmentation pipeline. This work proposes a method as a part of the LiTS (Liver Tumor Segmentation Challenge) competition for ISBI 17 and MICCAI 17 comparing methods for automatics egmentation of liver lesions in CT scans. By utilizing cascaded, densely connected 2D U-Nets and a Tversky-coefficient based loss function, our framework achieves very good shape extractions with high detection sensitivity, with competitive scores …
University of Science and Technology of China（中国科学技术大学）
Universal Lesion Detection (ULD) in computed tomography plays an essential role in computer-aided diagnosis. Promising ULD results have been reported by multi-slice-input detection approaches which model 3D context from multiple adjacent CT slices, but such methods still experience difficulty in obtaining a global representation among different slices and within each individual slice since they only use convolution-based fusion operations. In this paper, we propose a novel Slice Attention Transformer (SATr) block which can be easily plugged into convolution-based ULD backbones to form hybrid n…
Sberbank AI Laboratory（俄罗斯联邦储蓄银行人工智能实验室）；New York University（纽约大学）
Analysis of chest CT scans can be used in detecting parts of lungs that are affected by infectious diseases such as COVID-19.Determining the volume of lungs affected by lesions is essential for formulating treatment recommendations and prioritizingpatients by severity of the disease. In this paper we adopted an approach based on using an ensemble of deep convolutionalneural networks for segmentation of slices of lung CT scans. Using our models we are able to segment the lesions, evaluatepatients dynamics, estimate relative volume of lungs affected by lesions and evaluate the lung damage stage.…
Technische Universität Kaiserslautern（凯泽斯劳滕工业大学）；German Research Center for Artificial Intelligence GmbH (DFKI)（德国人工智能研究中心）
It is generally believed that the human visual system is biased towards the recognition of shapes rather than textures. This assumption has led to a growing body of work aiming to align deep models' decision-making processes with the fundamental properties of human vision. The reliance on shape features is primarily expected to improve the robustness of these models under covariate shift. In this paper, we revisit the significance of shape-biases for the classification of skin lesion images. Our analysis shows that different skin lesion datasets exhibit varying biases towards individual image …
Liver lesion segmentation is an important step for liver cancer diagnosis, treatment planning and treatment evaluation. LiTS (Liver Tumor Segmentation Challenge) provides a common testbed for comparing different automatic liver lesion segmentation methods. We participate in this challenge by developing a deep convolutional neural network (DCNN) method. The particular DCNN model works in 2.5D in that it takes a stack of adjacent slices as input and produces the segmentation map corresponding to the center slice. The model has 32 layers in total and makes use of both long range concatenation con…
CenterMamba-SAM: Center-Prioritized Scanning and Temporal Prototypes for Brain Lesion Segmentation
Northeastern University, Boston, United States（东北大学）；Stevens Institute of Technology, Hoboken, United States（史蒂文斯理工学院）；Rutgers University, Newark, United States（罗格斯大学）；Lehigh University, Bethlehem, United States（莱恩大学）；University of the Cumberlands, Williamsburg, United States（坎伯兰大学）
Focalyx Technologies（Focalyx科技公司）；City University of Hong Kong（香港城市大学）；University College London（伦敦大学学院）；Urological Research Network（泌尿外科研究网络）
Prostate cancer (PCa) is one of the leading causes of death for men worldwide. Multi-parametric magnetic resonance (mpMR) imaging has emerged as a non-invasive diagnostic tool for detecting and localising prostate tumours by specialised radiologists. These radiological examinations, for example, for differentiating malignant lesions from benign prostatic hyperplasia in transition zones and for defining the boundaries of clinically significant cancer, remain challenging and highly skill-and-experience-dependent. We first investigate experimental results in developing object detection neural net…
Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）深圳大数据研究院）；Sun Yat-sen University（中山大学）；Pazhou Lab（琶洲实验室）
Deep neural networks (DNNs) have been widely adopted in brain lesion detection and segmentation. However, locating small lesions in 2D MRI slices is challenging, and requires to balance between the granularity of 3D context aggregation and the computational complexity. In this paper, we propose a novel view-disentangled transformer to enhance the extraction of MRI features for more accurate tumour detection. First, the proposed transformer harvests long-range correlation among different positions in a 3D brain scan. Second, the transformer models a stack of slice features as multiple 2D views …
Deep transfer learning using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has shown strong predictive power in characterization of breast lesions. However, pretrained convolutional neural networks (CNNs) require 2D inputs, limiting the ability to exploit the rich 4D (volumetric and temporal) image information inherent in DCE-MRI that is clinically valuable for lesion assessment. Training 3D CNNs from scratch, a common method to utilize high-dimensional information in medical images, is computationally expensive and is not best suited for moderately sized healthcare datasets. …
MedOpenClaw and MedFlowBench: Auditing Medical Agents in Full-Study Workflows
Technical University of Munich（慕尼黑技术大学）；TUM University Hospital（TUM大学医院）；LMU Munich（慕尼黑大学）；Imperial College London（伦敦帝国理工学院）；University of Oxford（牛津大学）；Carnegie Mellon University（卡内基梅隆大学）；NVIDIA（NVIDIA公司）；National University of Singapore（新加坡国立大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
University of Electronic Science and Technology of China（电子科技大学）；University of Maryland School of Medicine（马里兰大学医学院）；Rutgers, the State University of New Jersey（罗格斯大学（新泽西州立大学））
Computed Tomography (CT) plays an important role in monitoring radiation-induced Pulmonary Fibrosis (PF), where accurate segmentation of the PF lesions is highly desired for diagnosis and treatment follow-up. However, the task is challenged by ambiguous boundary, irregular shape, various position and size of the lesions, as well as the difficulty in acquiring a large set of annotated volumetric images for training. To overcome these problems, we propose a novel convolutional neural network called PF-Net and incorporate it into a semi-supervised learning framework based on Iterative Confidence-…
EPRA U-Net: An Efficient Pyramid Residual Attention Framework for Accurate Infarct Segmentation in Diffusion-Weighted MRI
Yozgat Bozok University（亚兹加特博兹克大学）；Izmir Bakircay University（伊兹米尔巴克伊大学）；Queen Mary’s Digital Environment Research Institute (DERI)（皇后玛丽数字环境研究 institute (DERI)）；Firat University（费拉特大学）；Qatar University（卡塔尔大学）
Optimization in Sparse 2D to Dense 3D Weakly Supervised Learning: Application to Multi-Label Segmentation of Large ex vivo MRI Data
NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal（神经多极实验室，生物医学工程学院，蒙特利尔理工学院）；École Centrale de Lyon（里昂中央理工学院）；Mila - Quebec AI Institute（魁北克人工智能研究所）；Functional Neuroimaging Unit, CRIUGM, University of Montreal（功能神经影像单元，CRIUGM，蒙特利尔大学）；Translational Neuroradiology Section, National Institute of Neurological Disorders and Stroke, National Institutes of Health（转化神经放射学部门，国家神经疾病与中风研究所，国家卫生研究院）；Translational Imaging in Neurology (ThINk) Basel, Department of Biomedical Engineering, Faculty of Medicine, University Hospital Basel and University of Basel（神经学转化成像（ThINk）巴塞尔，生物医学工程系，医学院，巴塞尔大学医院和巴塞尔大学）；Neurologic Clinic and Policlinic, Departments of Medicine, University Hospital Basel, Switzerland（神经科诊所和多科诊所，医学院，巴塞尔大学医院，瑞士）；Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Switzerland（临床神经免疫学和神经科学巴塞尔研究中心（RC2NB），巴塞尔大学医院和巴塞尔大学，瑞士）；National Institute of Neurological Disorders and Stroke, National Institutes of Health（国家神经疾病与中风研究所，国家卫生研究院）；Centre de recherche du CHU Sainte-Justine, Université de Montréal, Montreal, QC, Canada（圣朱斯特医院研究中心，蒙特利尔大学，蒙特利尔，魁北克，加拿大）；Quantitative MRI core facility, NINDS, NIH（定量MRI核心设施，NINDS，NIH）；Experimental Immunotherapeutics Unit, Division of Neuroimmunology and Neurovirology, NINDS, NIH（实验免疫治疗单元，神经免疫学和神经病毒学部门，NINDS，NIH）
University of Toronto（多伦多大学）；The Hospital for Sick Children（SickKids医院（多伦多儿童医院））；Vector Institute（向量研究所）
Modern deep unsupervised learning methods have shown great promise for detecting diseases across a variety of medical imaging modalities. While previous generative modeling approaches successfully perform anomaly detection by learning the distribution of healthy 2D image slices, they process such slices independently and ignore the fact that they are correlated, all being sampled from a 3D volume. We show that incorporating the 3D context and processing whole-body MRI volumes is beneficial to distinguishing anomalies from their benign counterparts. In our work, we introduce a multi-channel sli…
Magnetic Resonance Imaging (MRI) is widely used for screening and staging prostate cancer. However, many prostate cancers have subtle features which are not easily identifiable on MRI, resulting in missed diagnoses and alarming variability in radiologist interpretation. Machine learning models have been developed in an effort to improve cancer identification, but current models localize cancer using MRI-derived features, while failing to consider the disease pathology characteristics observed on resected tissue. In this paper, we propose CorrSigNet, an automated two-step model that localizes p…
Accurate liver and lesion segmentation from computed tomography (CT) images are highly demanded in clinical practice for assisting the diagnosis and assessment of hepatic tumor disease. However, automatic liver and lesion segmentation from contrast-enhanced CT volumes is extremely challenging due to the diversity in contrast, resolution, and quality of images. Previous methods based on UNet for 2D slice-by-slice or 3D volume-by-volume segmentation either lack sufficient spatial contexts or suffer from high GPU computational cost, which limits the performance. To tackle these issues, we propose…
Sharif University of Technology（谢里夫理工大学）；Iranian Brain Mapping Biobank（伊朗脑图谱生物库）；National Brain Mapping Laboratory（国家脑图谱实验室）；Virtual University of Medical Sciences（虚拟医科大学）
Early detection of dementia through specific biomarkers in MR images plays a critical role in developing support strategies proactively. Fazekas scale facilitates an accurate quantitative assessment of the severity of white matter lesions and hence the disease. Imaging Biomarkers of dementia are multiple and comprehensive documentation of them is time-consuming. Therefore, any effort to automatically extract these biomarkers will be of clinical value while reducing inter-rater discrepancies. To tackle this problem, we propose to classify the disease severity based on the Fazekas scale through …
Unsupervised Detection of Post-Stroke Brain Abnormalities
Univ Rennes（里昂大学）；Inria（法国国家信息与自动化技术研究院）；CNRS（法国国家科学研究中心）；Inserm（法国国家医学研究院）；IRISA UMR 6074（里昂信息科学与自动化研究院）；CHU Rennes（里昂大学医院）；Siemens Healthineers（西门子医疗）；Centre de Kerpape（凯帕普中心）
