Applying deep learning methods to mammography assessment has remained a challenging topic. Dense noise with sparse expressions, mega-pixel raw data resolution, lack of diverse examples have all been factors affecting performance. The lack of pixel-level ground truths have especially limited segmentation methods in pushing beyond approximately bounding regions. We propose a classification approach grounded in high performance tissue assessment as an alternative to all-in-one localization and assessment models that is also capable of pinpointing the causal pixels. First, the objective of the mammography assessment task is formalized in the context of local tissue classifiers. Then, the accuracy of a convolutional neural net is evaluated on classifying patches of tissue with suspicious findings at varying scales, where highest obtained AUC is above $0.9$. The local evaluations of one such expert tissue classifier is used to augment the results of a heatmap regression model and additionally recover the exact causal regions at high resolution as a saliency image suitable for clinical settings.
NYU Langone Health（纽约大学朗格尼健康中心）；Jagiellonian University（雅盖隆大学）；NYU Abu Dhabi（纽约大学阿布扎比分校）；New York University（纽约大学）；Courant Institute, New York University（纽约大学库朗研究所）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）
Artificial intelligence (AI) is showing promise in improving clinical diagnosis. In breast cancer screening, recent studies show that AI has the potential to improve early cancer diagnosis and reduce unnecessary workup. As the number of proposed models and their complexity grows, it is becoming increasingly difficult to re-implement them. To enable reproducibility of research and to enable comparison between different methods, we release a meta-repository containing models for classification of screening mammograms. This meta-repository creates a framework that enables the evaluation of AI mod…
Carleton University（卡尔顿大学）；National Research Council Canada（加拿大国家研究委员会）
Breast cancer is the most common cancer in women worldwide. The most common screening technology is mammography. To reduce the cost and workload of radiologists, we propose a computer aided detection approach for classifying and localizing calcifications and masses in mammogram images. To improve on conventional approaches, we apply deep convolutional neural networks (CNN) for automatic feature learning and classifier building. In computer-aided mammography, deep CNN classifiers cannot be trained directly on full mammogram images because of the loss of image details from resizing at input laye…
Escola Politécnica, Universidade de São Paulo（圣保罗大学理工学院）；Instituto do Câncer do Estado de São Paulo（圣保罗州癌症研究所）；Faculdade de Medicina, Universidade de São Paulo（圣保罗大学医学院）
Some recent studies have described deep convolutional neural networks to diagnose breast cancer in mammograms with similar or even superior performance to that of human experts. One of the best techniques does two transfer learnings: the first uses a model trained on natural images to create a "patch classifier" that categorizes small subimages; the second uses the patch classifier to scan the whole mammogram and create the "single-view whole-image classifier". We propose to make a third transfer learning to obtain a "two-view classifier" to use the two mammographic views: bilateral craniocaud…
NYU Center for Data Science（纽约大学数据科学中心）；Courant Institute of Mathematical Sciences, NYU（纽约大学库朗数学科学研究所）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Jagiellonian University（雅盖隆大学）
In the last few years, deep learning classifiers have shown promising results in image-based medical diagnosis. However, interpreting the outputs of these models remains a challenge. In cancer diagnosis, interpretability can be achieved by localizing the region of the input image responsible for the output, i.e. the location of a lesion. Alternatively, segmentation or detection models can be trained with pixel-wise annotations indicating the locations of malignant lesions. Unfortunately, acquiring such labels is labor-intensive and requires medical expertise. To overcome this difficulty, weakl…
The rapid development of deep learning, a family of machine learning techniques, has spurred much interest in its application to medical imaging problems. Here, we develop a deep learning algorithm that can accurately detect breast cancer on screening mammograms using an "end-to-end" training approach that efficiently leverages training datasets with either complete clinical annotation or only the cancer status (label) of the whole image. In this approach, lesion annotations are required only in the initial training stage, and subsequent stages require only image-level labels, eliminating the …
Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, RadboudUMC Nijmegen（拉德堡德大学医学中心放射与核医学科诊断影像分析组）
We investigate the addition of symmetry and temporal context information to a deep Convolutional Neural Network (CNN) with the purpose of detecting malignant soft tissue lesions in mammography. We employ a simple linear mapping that takes the location of a mass candidate and maps it to either the contra-lateral or prior mammogram and Regions Of Interest (ROI) are extracted around each location. We subsequently explore two different architectures (1) a fusion model employing two datastreams were both ROIs are fed to the network during training and testing and (2) a stage-wise approach where a s…
Mammographic Breast Positioning Assessment via Deep Learning
Istanbul Technical University（伊斯坦布尔理工大学）；Acibadem Healthcare Group（阿西巴德姆医疗集团）；Acibadem Mehmet Ali Aydinlar University（阿西巴德姆穆罕默德·阿里·艾丁拉尔大学）；Sisli Hamidiye Etfal Training and Research Hospital（锡什利哈米迪耶埃特法尔培训与研究医院）
Advanced deep learning (DL) algorithms may predict the patient's risk of developing breast cancer based on the Breast Imaging Reporting and Data System (BI-RADS) and density standards. Recent studies have suggested that the combination of multi-view analysis improved the overall breast exam classification. In this paper, we propose a novel multi-view DL approach for BI-RADS and density assessment of mammograms. The proposed approach first deploys deep convolutional networks for feature extraction on each view separately. The extracted features are then stacked and fed into a Light Gradient Boo…
Impact of loss functions on the performance of a deep neural network designed to restore low-dose digital mammography
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Fudan University（复旦大学）；Shanghai Center for Brain Science and Brain-inspired Technology（上海脑科学与类脑研究中心）；São Carlos School of Engineering, University of São Paulo（圣保罗大学圣卡洛斯工程学院）；Ribeirão Preto School of Medicine, University of São Paulo（圣保罗大学里贝朗普雷图医学院）
LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol
Department of Radiology, Northwestern University（北western大学放射科）；Department of Radiation Oncology, University of Health Sciences Prof. Dr. Cemil Tascioglu City Hospital（健康科学大学教授Dr. Cemil Tascioglu医院放射肿瘤科）；Department of Radiology, Istanbul University（伊斯坦布尔大学放射科）
Bias and Generalizability of Foundation Models across Datasets in Breast Mammography
University Hospital Bonn（波恩大学医院）；Technical University of Munich（慕尼黑技术大学）；Lebanese Hospital Geitaoui（黎巴嫩Geitaoui医院）；Helmholtz AI, Helmholtz Munich（海德堡人工智能，海德堡慕尼黑）
NYU Langone Health（纽约大学朗格尼医学中心）；Jagiellonian University（雅盖隆大学）；New York University（纽约大学）
3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple …
Mammography is the most widely used gold standard for screening breast cancer, where, mass detection is considered as the prominent step. Detecting mass in the breast is, however, an arduous problem as they usually have large variations between them in terms of shape, size, boundary, and texture. In this literature, the process of mass detection is automated with the use of transfer learning techniques of Deep Convolutional Neural Networks (DCNN). Pre-trained VGG19 network is used to extract features which are then followed by bagged decision tree for features selection and then a Support Vect…
Extracting discriminative features plays a crucial role in the fine-grained visual classification task. Most of the existing methods focus on developing attention or augmentation mechanisms to achieve this goal. However, addressing the ambiguity in the top-k prediction classes is not fully investigated. In this paper, we introduce a Self Assessment Classifier, which simultaneously leverages the representation of the image and top-k prediction classes to reassess the classification results. Our method is inspired by continual learning with coarse-grained and fine-grained classifiers to increase…
Multivariate Analysis on Performance Gaps of Artificial Intelligence Models in Screening Mammography
Kennesaw State University（肯尼索州立大学）；Emory University（埃默里大学）；Washington University in St. Louis School of Medicine（圣路易斯华盛顿大学医学院）；Mayo Clinic Arizona（梅奥诊所亚利桑那分院）；Arizona State University（亚利桑那州立大学）；York University（约克大学）
University of Colorado, Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Image classification in the open-world must handle out-of-distribution (OOD) images. Systems should ideally reject OOD images, or they will map atop of known classes and reduce reliability. Using open-set classifiers that can reject OOD inputs can help. However, optimal accuracy of open-set classifiers depend on the frequency of OOD data. Thus, for either standard or open-set classifiers, it is important to be able to determine when the world changes and increasing OOD inputs will result in reduced system reliability. However, during operations, we cannot directly assess accuracy as there are …
Dataset complexity assessment aims to predict classification performance on a dataset with complexity calculation before training a classifier, which can also be used for classifier selection and dataset reduction. The training process of deep convolutional neural networks (DCNNs) is iterative and time-consuming because of hyperparameter uncertainty and the domain shift introduced by different datasets. Hence, it is meaningful to predict classification performance by assessing the complexity of datasets effectively before training DCNN models. This paper proposes a novel method called cumulati…
Center for Data Science, New York University（纽约大学数据科学中心）；Department of Computer Science, Courant Institute, New York University（纽约大学库朗研究所计算机科学系）；Department of Radiology, NYU School of Medicine（纽约大学医学院放射科）；Perlmutter Cancer Center, NYU Langone Health（纽约大学朗格尼健康系统珀尔马特癌症中心）；Center for Advanced Imaging Innovation and Research, NYU Langone Health（纽约大学朗格尼健康系统先进成像创新与研究中心）；CIFAR Associate Fellow（CIFAR研究员（注：CIFAR是加拿大英属哥伦比亚大学的一个研究机构，但根据提取规则，此处仅提取“CIFAR”相关部分，严格按英文保留））
Medical images differ from natural images in significantly higher resolutions and smaller regions of interest. Because of these differences, neural network architectures that work well for natural images might not be applicable to medical image analysis. In this work, we extend the globally-aware multiple instance classifier, a framework we proposed to address these unique properties of medical images. This model first uses a low-capacity, yet memory-efficient, network on the whole image to identify the most informative regions. It then applies another higher-capacity network to collect detail…
Disaggregated performance metrics across demographic groups are a hallmark of fairness assessments in computer vision. These metrics successfully incentivized performance improvements on person-centric tasks such as face analysis and are used to understand risks of modern models. However, there is a lack of discussion on the vulnerabilities of these measurements for more complex computer vision tasks. In this paper, we consider multi-label image classification and, specifically, object categorization tasks. First, we highlight design choices and trade-offs for measurement that involve more nua…
Multi-parametric magnetic resonance imaging (mpMRI) has a growing role in detecting prostate cancer lesions. Thus, it is pertinent that medical professionals who interpret these scans reduce the risk of human error by using computer-aided detection systems. The variety of algorithms used in system implementation, however, has yielded mixed results. Here we investigate the best machine learning classifier for each prostate zone. We also discover salient features to clarify the models' classification rationale. Of the data provided, we gathered and augmented T2 weighted images and apparent diffu…
University of Pennsylvania（宾夕法尼亚大学）；Mayo Clinic（梅奥诊所）
Breast density is an important risk factor for breast cancer that also affects the specificity and sensitivity of screening mammography. Current federal legislation mandates reporting of breast density for all women undergoing breast screening. Clinically, breast density is assessed visually using the American College of Radiology Breast Imaging Reporting And Data System (BI-RADS) scale. Here, we introduce an artificial intelligence (AI) method to estimate breast percentage density (PD) from digital mammograms. Our method leverages deep learning (DL) using two convolutional neural network arch…
University of Nevada, Reno（内华达大学里诺分校）；Advanced Robotics and Automation (ARA) Laboratory（先进机器人与自动化（ARA）实验室）
Accurate breast cancer diagnosis through mammography has the potential to save millions of lives around the world. Deep learning (DL) methods have shown to be very effective for mass detection in mammograms. Additional improvements of current DL models will further improve the effectiveness of these methods. A critical issue in this context is how to pick the right hyperparameters for DL models. In this paper, we present GA-E2E, a new approach for tuning the hyperparameters of DL models for brest cancer detection using Genetic Algorithms (GAs). Our findings reveal that differences in parameter…
Screening mammography is an important front-line tool for the early detection of breast cancer, and some 39 million exams are conducted each year in the United States alone. Here, we describe a multi-scale convolutional neural network (CNN) trained with a curriculum learning strategy that achieves high levels of accuracy in classifying mammograms. Specifically, we first train CNN-based patch classifiers on segmentation masks of lesions in mammograms, and then use the learned features to initialize a scanning-based model that renders a decision on the whole image, trained end-to-end on outcome …
Beyond Perceptual Distance: Discrepancy Assessment on Deep Representation for Out-of-Distribution Detection with Diffusion Model
Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系）；Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University（香港理工大学电子与电气工程系）；The Intsig Information Co., Ltd., Shanghai, China（上海Intsig信息有限公司）；School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）
Synthesizing high resolution photorealistic images has been a long-standing challenge in machine learning. In this paper we introduce new methods for the improved training of generative adversarial networks (GANs) for image synthesis. We construct a variant of GANs employing label conditioning that results in 128x128 resolution image samples exhibiting global coherence. We expand on previous work for image quality assessment to provide two new analyses for assessing the discriminability and diversity of samples from class-conditional image synthesis models. These analyses demonstrate that high…
Hyperspectral Imaging-Based Grain Quality Assessment With Limited Labelled Data
School of Information Technology, Deakin University（迪肯大学信息技术学院）；Institute of Innovation, Science and Sustainability, Federation University（联邦大学创新、科学与可持续发展研究所）
Duke University（杜克大学）；Duke Clinical Research Institute（杜克临床研究所）；Duke-Margolis Center for Health Policy（杜克-马戈利斯卫生政策中心）；Duke Cancer Institute（杜克癌症研究所）
The assessment of knee osteoarthritis (KOA) severity on knee X-rays is a central criteria for the use of total knee arthroplasty. However, this assessment suffers from imprecise standards and a remarkably high inter-reader variability. An algorithmic, automated assessment of KOA severity could improve overall outcomes of knee replacement procedures by increasing the appropriateness of its use. We propose a novel deep learning-based five-step algorithm to automatically grade KOA from posterior-anterior (PA) views of radiographs: (1) image preprocessing (2) localization of knees joints in the im…
When parameterizing vehicle control algorithms for stability or trajectory control, the road-tire friction coefficient is an essential model parameter when it comes to control performance. One major impact on the friction coefficient is the condition of the road surface. A camera-based, forward-looking classification of the road-surface helps enabling an early parametrization of vehicle control algorithms. In this paper, we train and compare two different Deep Convolutional Neural Network models, regarding their application for road friction estimation and describe the challenges for training …
VUNO Inc.（VUNO公司）；University of Ulsan, College of Medicine（蔚山大学医学院）；Asan Medical Center（峨山医学中心）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This study presents a novel approach to bone age assessment (BAA) using a multi-view, multi-task classification model based on the Sauvegrain method. A straightforward solution to automating the Sauvegrain method, which assesses a maturity score for each landmark in the elbow and predicts the bone age, is to train classifiers independently to score each region of interest (RoI), but this approach limits the accessible information to local morphologies and increases computational costs. As a result, this work proposes a self-accumulative vision transformer (SAT) that mitigates anisotropic behav…
GFE-Mamba: Mamba-based AD Multi-modal Progression Assessment via Generative Feature Extraction from MCI
Hangzhou Dianzi University（杭州电子科技大学）；Beihang University（北京航空航天大学）；Xuanwu Hospital, Capital Medical University（首都医科大学宣武医院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
The Harker School（哈克学校）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）
Current analysis of tumor proliferation, the most salient prognostic biomarker for invasive breast cancer, is limited to subjective mitosis counting by pathologists in localized regions of tissue images. This study presents the first data-driven integrative approach to characterize the severity of tumor growth and spread on a categorical and molecular level, utilizing multiple biologically salient deep learning classifiers to develop a comprehensive prognostic model. Our approach achieves pathologist-level performance on three-class categorical tumor severity prediction. It additionally pionee…
University of Copenhagen（哥本哈根大学）；Hvidovre Hospital（维德奥夫医院）；Bispebjerg Hospital（比斯珀比约医院）；Erasmus MC - University Medical Center Rotterdam（伊拉斯姆斯医学中心——鹿特丹大学医学中心）
Accurate assessment of pulmonary emphysema is crucial to assess disease severity and subtype, to monitor disease progression and to predict lung cancer risk. However, visual assessment is time-consuming and subject to substantial inter-rater variability and standard densitometry approaches to quantify emphysema remain inferior to visual scoring. We explore if machine learning methods that learn from a large dataset of visually assessed CT scans can provide accurate estimates of emphysema extent. We further investigate if machine learning algorithms that learn from a scoring of emphysema extent…
