Interpretable deep learning regression for breast density estimation on MRI
Image Sciences Institute, University Medical Center Utrecht, Utrecht University（乌得勒支大学乌得勒支大学医学中心影像科学研究所）；Netherlands Cancer Institute – Antoni van Leeuwenhoek Hospital（荷兰癌症研究所——安东尼·范·列文虎克医院）
Bas H. M. van der Velden, Max A. A. Ragusi, Markus H. A. Janse, Claudette E. Loo, Kenneth G. A. Gilhuijs
备注：
This paper has been published as: Van der Velden, B.H.M., Ragusi, M.A.A., Janse, M.H.A., Loo, C.E., Gilhuijs, K.G.A. "Interpretable deep learning regression for breast density estimation on MRI." Medical Imaging 2020: Computer-Aided Diagnosis. Vol. 11314. International Society for Optics and Photonics, 2020
Breast density, which is the ratio between fibroglandular tissue (FGT) and total breast volume, can be assessed qualitatively by radiologists and quantitatively by computer algorithms. These algorithms often rely on segmentation of breast and FGT volume. In this study, we propose a method to directly assess breast density on MRI, and provide interpretations of these assessments.
We assessed breast density in 506 patients with breast cancer using a regression convolutional neural network (CNN). The input for the CNN were slices of breast MRI of 128 x 128 voxels, and the output was a continuous density value between 0 (fatty breast) and 1 (dense breast). We used 350 patients to train the CNN, 75 for validation, and 81 for independent testing. We investigated why the CNN came to its predicted density using Deep SHapley Additive exPlanations (SHAP).
The density predicted by the CNN on the testing set was significantly correlated with the ground truth densities (N = 81 patients, Spearman's rho = 0.86, P < 0.001). When inspecting what the CNN based its predictions on, we found that voxels in FGT commonly had positive SHAP-values, voxels in fatty tissue commonly had negative SHAP-values, and voxels in non-breast tissue commonly had SHAP-values near zero. This means that the prediction of density is based on the structures we expect it to be based on, namely FGT and fatty tissue.
To conclude, we presented an interpretable deep learning regression method for breast density estimation on MRI with promising results.
School of Information Technology and Electrical Engineering, University of Queensland（昆士兰大学信息技术与电气工程学院）；Science and Engineering Faculty, Queensland University of Technology（昆士兰科技大学科学与工程学院）
Previous studies on computer aided detection/diagnosis (CAD) in 4D breast magnetic resonance imaging (MRI) regard lesion detection, segmentation and characterization as separate tasks, and typically require users to manually select 2D MRI slices or regions of interest as the input. In this work, we present a breast MRI CAD system that can handle 4D multimodal breast MRI data, and integrate lesion detection, segmentation and characterization with no user intervention. The proposed CAD system consists of three major stages: region candidate generation, feature extraction and region candidate cla…
The Chinese University of Hong Kong（香港中文大学）；Imsight Medical Technology, Co., Ltd.（推想医疗科技股份有限公司）；Imperial College London（伦敦帝国理工学院）；The Fifth Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第五医学中心）；Beijing Image Diagnostic Center of Rimag（北京一脉阳光影像诊断中心）
Accurate and automatic analysis of breast MRI plays an important role in early diagnosis and successful treatment planning for breast cancer. Due to the heterogeneity nature, accurate diagnosis of tumors remains a challenging task. In this paper, we propose to identify breast tumor in MRI by Cosine Margin Sigmoid Loss (CMSL) with deep learning (DL) and localize possible cancer lesion by COrrelation Attention Map (COAM) based on the learned features. The CMSL embeds tumor features onto a hypersphere and imposes a decision margin through cosine constraints. In this way, the DL model could learn …
BreastSegNet: Multi-label Segmentation of Breast MRI
Department of Biostatistics \& Bioinformatics, Duke University Department of Electrical；Computer Engineering, Duke University Department of Radiology, Duke University Medical Center Department of Computer Science, Duke University
Predictive Enhancement Calibration for Latent Breast MRI Virtual Contrast Enhancement
The First Affiliated Hospital of Chongqing Medical University（重庆医科大学附属第一医院）；Chongqing Municipal Health Commission（重庆市卫生健康委员会）；Chongqing Translational Medicine Center（重庆市转化医学中心）
Pre- to Post-Contrast Breast MRI Synthesis for Enhanced Tumour Segmentation
Universitat de Barcelona（巴塞罗那大学）；Helmholtz Center Munich（慕尼黑亥姆霍兹中心）；Technical University of Munich（慕尼黑工业大学）；Karolinska Institutet（卡罗林斯卡学院）；King’s College London（伦敦国王学院）
Synthesis of Contrast-Enhanced Breast MRI Using Multi-b-Value DWI-based Hierarchical Fusion Network with Attention Mechanism
Netherlands Cancer Institute (NKI)（荷兰癌症研究所）；Maastricht University（马斯特里赫特大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Fondazione Universitaria A. Gemelli, IRCCS Roma（罗马A.杰梅利大学基金会IRCCS）
Accurate segmentation of breast lesions is a crucial step in evaluating the characteristics of tumors. However, this is a challenging task, since breast lesions have sophisticated shape, topological structure, and variation in the intensity distribution. In this paper, we evaluated the performance of three unsupervised algorithms for the task of breast Magnetic Resonance (MRI) lesion segmentation, namely, Gaussian Mixture Model clustering, K-means clustering and a marker-controlled Watershed transformation based method. All methods were applied on breast MRI slices following selection of regio…
Iran University of Science and Technology（伊朗科技大学）；Radboud University Medical Center（拉德堡德大学医学中心）；Netherlands Cancer Institute（荷兰癌症研究所）
Purpose: We propose a deep learning-based computer-aided detection (CADe) method to detect breast lesions in ultrafast DCE-MRI sequences. This method uses both the three-dimensional spatial information and temporal information obtained from the early-phase of the dynamic acquisition. Methods: The proposed CADe method, based on a modified 3D RetinaNet model, operates on ultrafast T1 weighted sequences, which are preprocessed for motion compensation, temporal normalization, and are cropped before passing into the model. The model is optimized to enable the detection of relatively small breast le…
GuidedMorph: Two-Stage Deformable Registration for Breast MRI
Department of Electrical and Computer Engineering, Duke University（电子工程系，杜克大学）；Department of Biostatistics & Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida（健康结果与生物医学信息学系，医学院，佛罗里达大学）
Early prediction of pathological complete response (pCR) following neoadjuvant chemotherapy (NAC) for breast cancer plays a critical role in surgical planning and optimizing treatment strategies. Recently, machine and deep-learning based methods were suggested for early pCR prediction from multi-parametric MRI (mp-MRI) data including dynamic contrast-enhanced MRI and diffusion-weighted MRI (DWI) with moderate success. We introduce PD-DWI, a physiologically decomposed DWI machine-learning model to predict pCR from DWI and clinical data. Our model first decomposes the raw DWI data into the vario…
Diffeomorphic Multi-Resolution Deep Learning Registration for Applications in Breast MRI
Auckland Bioengineering Institute（奥克兰生物工程研究所）；University of Auckland（奥克兰大学）；Faculty of Medical and Health Sciences（医学与健康科学学院）；MGH/HST Martinos Center for Biomedical Imaging（麻省总医院/哈佛健康科学技术生物医学成像马蒂诺斯中心）；Harvard Medical School（哈佛医学院）；MIT（麻省理工学院）
SAFE-Diff: Scale-Aware Attention and Feature-Dispersive Diffusion with Uncertainty Estimation for Contrast-Enhanced Breast MRI Synthesis
Department of Medical Imaging, Radboud University Medical Center（鲁文大学医学中心医学影像部）；Department of Radiology, Netherlands Cancer Institute（荷兰癌症研究所放射科）；Maastro Clinic（马斯垂克诊所）；Faculty of Applied Science, Macao Polytechnic University（澳门理工大学应用科学学院）；Department of Radiation Oncology, Netherlands Cancer Institute（荷兰癌症研究所放射肿瘤科）
The University of Adelaide（阿德莱德大学）；Queensland University of Technology（昆士兰科技大学）；Instituto Superior Tecnico（里斯本高等理工学院）
We propose a new method for breast cancer screening from DCE-MRI based on a post-hoc approach that is trained using weakly annotated data (i.e., labels are available only at the image level without any lesion delineation). Our proposed post-hoc method automatically diagnosis the whole volume and, for positive cases, it localizes the malignant lesions that led to such diagnosis. Conversely, traditional approaches follow a pre-hoc approach that initially localises suspicious areas that are subsequently classified to establish the breast malignancy -- this approach is trained using strongly annot…
Evaluating ADC-only deep learning pipelines for breast cancer detection and segmentation using standalone diffusion-weighted MRI
University of Oviedo（奥维耶多大学）；University of A Coruña（拉科鲁尼亚大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Purdue University（普渡大学）；Asturias Central University Hospital (HUCA)（阿斯图里亚斯中央大学医院）
A Large Model for Non-invasive and Personalized Management of Breast Cancer from Multiparametric MRI
The Hong Kong University of Science and Technology（香港科技大学）；PLA Middle Military Command General Hospital（中国人民解放军中部战区总医院）；The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）；Yunnan Cancer Hospital（云南省肿瘤医院）；th Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第五医学中心）；Southern Medical University（南方医科大学）
Friedrich-Alexander-University Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；Universitat Politècnica de València（瓦伦西亚理工大学）；Universitätsklinikum Erlangen（埃尔朗根大学医院）
Magnetic resonance imaging (MRI) is an effective imaging modality for identifying and localizing breast lesions in women. Accurate and precise lesion segmentation using a computer-aided-diagnosis (CAD) system, is a crucial step in evaluating tumor volume and in the quantification of tumor characteristics. However, this is a challenging task, since breast lesions have sophisticated shape, topological structure, and high variance in their intensity distribution across patients. In this paper, we propose a novel marker-controlled watershed transformation-based approach, which uses the brightest p…
Temporal Neural Cellular Automata: Application to modeling of contrast enhancement in breast MRI
Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Germany School of Computation, Information；Technology, Technical University of Munich, Germany Departament de Matematiques i Informatica, Universitat de Barcelona, Spain Institució Catalana de Recerca i Estudis Avan c ats (ICREA), Barcelona, Spain Institute for Diagnostic；Interventional Radiology, School of Medicine \& Health, Klinkum Rechts der Isar, Technical University of Munich, Germany German Cancer Consortium (DKTK), Partner Site Munich, Germany School of Biomedical Engineering \& Imaging Sciences, King's College London, UK
University of Toronto（多伦多大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）
High predictive performance and ease of use and interpretability are important requirements for the applicability of a computer-aided diagnosis (CAD) to human reading studies. We propose a CAD system specifically designed to be more comprehensible to the radiologist reviewing screening breast MRI studies. Multiparametric imaging features are combined to produce a CAD system for differentiating cancerous and non-cancerous lesions. The complete system uses a rule-extraction algorithm to present lesion classification results in an easy to understand graph visualization.
TopoTxR: A topology-guided deep convolutional network for breast parenchyma learning on DCE-MRIs
State University of New York at Stony Brook（纽约州立大学石溪分校）；Newark Beth Israel Medical Center（纽瓦克贝斯以色列医疗中心）；Columbia University Irving Medical Center（哥伦比亚大学欧文医疗中心）；Emory University School of Medicine（埃默里大学医学院）；Georgia Institute of Technology（佐治亚理工学院）
Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI
School of Health Science and Engineering, University of Shanghai for Science and Technology（上海理工大学健康科学与工程学院）；School of Biomedical Engineering, ShanghaiTech University（上海科技大学生物医学工程学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Ruijin Hospital, Shanghai Jiaotong University School of Medicine（上海交通大学医学院附属瑞金医院）；Shanghai General Hospital（上海交通大学附属第一人民医院）
To deform or not: treatment-aware longitudinal registration for breast DCE-MRI during neoadjuvant chemotherapy via unsupervised keypoints detection
Radboud University Medical Centre（拉德堡德大学医学中心）；The Netherlands Cancer Institute（荷兰癌症研究所）；Macao Polytechnic University（澳门理工大学）；Maastricht University Medical Centre（马斯特里赫特大学医学中心）
MeisenMeister: A Simple Two Stage Pipeline for Breast Cancer Classification on MRI
German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Germany（德国癌症研究中心（DKFZ）海德堡，医学影像计算部，德国）；Medical Faculty, Heidelberg University, Germany（海德堡大学医学学院，德国）；Faculty of Mathematics and Computer Science, Heidelberg University（海德堡大学数学与计算机科学学院）；Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany（放射肿瘤学系学习组，海德堡大学医院，海德堡，德国）
LoGo-MR: Screening Breast MRI for Cancer Risk Prediction by Efficient Omni-Slice Modeling
Netherlands Cancer Institute（荷兰癌症研究所）；Radboud University Medical Center（拉德堡德大学医学中心）；University Medical Center Utrecht（乌得勒支大学医学中心）；Macao Polytechnic University（澳门理工大学）
The University of Adelaide（阿德莱德大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；Delft University of Technology（代尔夫特理工大学）；Queensland University of Technology（昆士兰科技大学）
There is a heated debate on how to interpret the decisions provided by deep learning models (DLM), where the main approaches rely on the visualization of salient regions to interpret the DLM classification process. However, these approaches generally fail to satisfy three conditions for the problem of lesion detection from medical images: 1) for images with lesions, all salient regions should represent lesions, 2) for images containing no lesions, no salient region should be produced,and 3) lesions are generally small with relatively smooth borders. We propose a new model-agnostic paradigm to …
PRISM-Net: Patient-specific reference-guided inter-breast symmetry matching for three-class breast DCE-MRI classification
Nankai University（南开大学）；Zhongguancun Academy（中关村学院）；The First Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第一医学中心）；Beijing University of Posts and Telecommunications（北京邮电大学）；The Six Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第六医学中心）；Tsinghua University（清华大学）
Tel Aviv University（特拉维夫大学）；Sheba Medical Center（示巴医疗中心）
Medical reports are an essential medium in recording a patient's condition throughout a clinical trial. They contain valuable information that can be extracted to generate a large labeled dataset needed for the development of clinical tools. However, the majority of medical reports are stored in an unregularized format, and a trained human annotator (typically a doctor) must manually assess and label each case, resulting in an expensive and time consuming procedure. In this work, we present a framework for developing a multilingual breast MRI report classifier using a custom-built language rep…
Mask-Guided Multi-Channel SwinUNETR Framework for Robust MRI Classification
Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain（巴塞罗那大学数学与计算机系）；Computer Vision Center, Bellaterra, Spain（贝尔特拉拉计算机视觉中心）；Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig Lluís Companys 23, Barcelona, Spain（加泰罗尼亚高级研究与教育研究所）
Deep transfer learning using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has shown strong predictive power in characterization of breast lesions. However, pretrained convolutional neural networks (CNNs) require 2D inputs, limiting the ability to exploit the rich 4D (volumetric and temporal) image information inherent in DCE-MRI that is clinically valuable for lesion assessment. Training 3D CNNs from scratch, a common method to utilize high-dimensional information in medical images, is computationally expensive and is not best suited for moderately sized healthcare datasets. …
The image acquisition parameters (IAPs) used to create MRI scans are central to defining the appearance of the images. Deep learning models trained on data acquired using certain parameters might not generalize well to images acquired with different parameters. Being able to recover such parameters directly from an image could help determine whether a deep learning model is applicable, and could assist with data harmonization and/or domain adaptation. Here, we introduce a neural network model that can predict many complex IAPs used to generate an MR image with high accuracy solely using the im…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；Department of Radiology, Antoni van Leeuwenhoek Hospital - The Netherlands Cancer Institute（荷兰癌症研究所安东尼·范·列文虎克医院放射科）
Response of breast cancer to neoadjuvant chemotherapy (NAC) can be monitored using the change in visible tumor on magnetic resonance imaging (MRI). In our current workflow, seed points are manually placed in areas of enhancement likely to contain cancer. A constrained volume growing method uses these manually placed seed points as input and generates a tumor segmentation. This method is rigorously validated using complete pathological embedding. In this study, we propose to exploit deep learning for fast and automatic seed point detection, replacing manual seed point placement in our existing …
Duke University（杜克大学）；Duke University Comprehensive Cancer Center（杜克大学综合癌症中心）
Purpose: To determine whether deep learning-based algorithms applied to breast MR images can aid in the prediction of occult invasive disease following the di- agnosis of ductal carcinoma in situ (DCIS) by core needle biopsy. Material and Methods: In this institutional review board-approved study, we analyzed dynamic contrast-enhanced fat-saturated T1-weighted MRI sequences of 131 patients at our institution with a core needle biopsy-confirmed diagnosis of DCIS. The patients had no preoperative therapy before breast MRI and no prior history of breast cancer. We explored two different deep lear…
SpurBreast: A Curated Dataset for Investigating Spurious Correlations in Real-world Breast MRI Classification
Center for Biosystems；Biotech Data Science, Ghent University Global Campus, Incheon, Republic of Korea IDLab, ELIS, Ghent University, Ghent, Belgium Department of Applied Mathematics, Computer Science；Statistics, Ghent University, Ghent, Belgium ( )
Case Western Reserve University（凯斯西储大学）；University of Mons（蒙斯大学）；Cleveland Clinic（克利夫兰诊所）；Yale School of Medicine（耶鲁医学院）；Institut Jules Bordet（朱尔斯·博尔代研究所）；University Hospitals Cleveland Medical Center（大学医院克利夫兰医学中心）；Louis Stokes Cleveland VA Medical Center（路易斯·斯托克斯克利夫兰退伍军人事务医疗中心）
Predicting response to neoadjuvant therapy is a vexing challenge in breast cancer. In this study, we evaluate the ability of deep learning to predict response to HER2-targeted neo-adjuvant chemotherapy (NAC) from pre-treatment dynamic contrast-enhanced (DCE) MRI acquired prior to treatment. In a retrospective study encompassing DCE-MRI data from a total of 157 HER2+ breast cancer patients from 5 institutions, we developed and validated a deep learning approach for predicting pathological complete response (pCR) to HER2-targeted NAC prior to treatment. 100 patients who received HER2-targeted ne…
The Netherlands Cancer Institute（荷兰癌症研究所）；Nanjing University of Information Science and Technology（南京信息工程大学）
Multi-sequence MRIs can be necessary for reliable diagnosis in clinical practice due to the complimentary information within sequences. However, redundant information exists across sequences, which interferes with mining efficient representations by modern machine learning or deep learning models. To handle various clinical scenarios, we propose a sequence-to-sequence generation framework (Seq2Seq) for imaging-differentiation representation learning. In this study, not only do we propose arbitrary 3D/4D sequence generation within one model to generate any specified target sequence, but also we…
Robustness of breast lesion segmentation under MRI undersampling improves with k-space-aware deep learning
Institute for AI in Medicine (IKIM), University Hospital Essen（人工智能医学研究所（IKIM），埃森大学医院）；Department of Physics, Technical University Dortmund（物理系，多特蒙德技术大学）；Division of Radiology, German Cancer Research Center (DKFZ)（放射学部，德国癌症研究中心（DKFZ））；Cancer Research Center Cologne Essen (CCCE), University Medicine Essen（科隆埃森癌症研究中心（CCCE），埃森大学医学中心）；RACOON Study Group, Site Essen（RACOON研究组，埃森站点）；German Cancer Consortium (DKTK), Partner Site Essen（德国癌症联合会（DKTK），埃森合作伙伴站点）；Medical Faculty and Faculty of Computer Science, University of Duisburg-Essen（医学系和计算机科学系，杜伊斯堡-埃森大学）
GRAPHITE: Graph-Based Interpretable Tissue Examination for Enhanced Explainability in Breast Cancer Histopathology
School of Computer Science and Engineering（计算机科学与工程学院）；University of New South Wales（新南威尔士大学）；St. George and Sutherland Clinical School（圣乔治与萨瑟兰临床学院）；Department of Anatomical Pathology（解剖病理学部）；NSW Health Pathology（新南威尔士州卫生病理学）；Cancer Care Centre（癌症护理中心）
Segmenting a MRI images into homogeneous texture regions representing disparate tissue types is often a useful preprocessing step in the computer-assisted detection of breast cancer. That is why we proposed new algorithm to detect cancer in mammogram breast cancer images. In this paper we proposed segmentation using vector quantization technique. Here we used Linde Buzo-Gray algorithm (LBG) for segmentation of MRI images. Initially a codebook of size 128 was generated for MRI images. These code vectors were further clustered in 8 clusters using same LBG algorithm. These 8 images were displayed…
Towards Affordable Tumor Segmentation and Visualization for 3D Breast MRI Using SAM2
Center for Biosystems and Biotech Data Science, Ghent University Global Campus, Incheon, Republic of Korea（生物系统与生物技术数据科学中心，格罗宁根大学全球校园，韩国Incheon）；Department of Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium（数学、计算机科学与统计学系，格罗宁根大学，比利时Ghent）；IDLab, Department of Electronics and Information Systems, Ghent University, Ghent, Belgium（IDLab，电子与信息系统系，格罗宁根大学，比利时Ghent）
Adapted Foundation Models for Breast MRI Triaging in Contrast-Enhanced and Non-Contrast Enhanced Protocols
Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（弗里德里希-亚历山大大学埃尔兰根-纽伦堡分校）；Uniklinikum Erlangen（埃尔兰根大学医院）；Radiology München（慕尼黑放射科）；Polish Academy of Science（波兰科学院）
LesiOnTime -- Joint Temporal and Clinical Modeling for Small Breast Lesion Segmentation in Longitudinal DCE-MRI
Computational Imaging Research Lab, Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Austria（医学影像研究实验室，生物医学影像与影像引导治疗系，维也纳医科大学）；Comprehensive Center for AI in Medicine, Medical University of Vienna, Austria（医学人工智能综合中心，维也纳医科大学）；Breast Health Center, Privatklinik Döbling, Austria（乳腺健康中心，多克林根私人诊所）；Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Austria（生物医学影像与影像引导治疗系，维也纳医科大学）；Department of Obstetrics and Gynecology, Medical University of Vienna, Austria（妇产科系，维也纳医科大学）；Department of Pathology, Medical University of Vienna, Austria（病理学系，维也纳医科大学）；Comprehensive Cancer Center, Medical University of Vienna, Austria（综合癌症中心，维也纳医科大学）
