Computer-aided detection or decision support systems aim to improve breast cancer screening programs by helping radiologists to evaluate digital mammography (DM) exams. Commonly such methods proceed in two steps: selection of candidate regions for malignancy, and later classification as either malignant or not. In this study, we present a candidate detection method based on deep learning to automatically detect and additionally segment soft tissue lesions in DM. A database of DM exams (mostly bilateral and two views) was collected from our institutional archive. In total, 7196 DM exams (28294 DM images) acquired with systems from three different vendors (General Electric, Siemens, Hologic) were collected, of which 2883 contained malignant lesions verified with histopathology. Data was randomly split on an exam level into training (50\%), validation (10\%) and testing (40\%) of deep neural network with u-net architecture. The u-net classifies the image but also provides lesion segmentation. Free receiver operating characteristic (FROC) analysis was used to evaluate the model, on an image and on an exam level. On an image level, a maximum sensitivity of 0.94 at 7.93 false positives (FP) per image was achieved. Similarly, per exam a maximum sensitivity of 0.98 at 7.81 FP per image was achieved. In conclusion, the method could be used as a candidate selection model with high accuracy and with the additional information of lesion segmentation.
The rapid development of deep learning, a family of machine learning techniques, has spurred much interest in its application to medical imaging problems. Here, we develop a deep learning algorithm that can accurately detect breast cancer on screening mammograms using an "end-to-end" training approach that efficiently leverages training datasets with either complete clinical annotation or only the cancer status (label) of the whole image. In this approach, lesion annotations are required only in the initial training stage, and subsequent stages require only image-level labels, eliminating the …
Hamburg University of Technology（汉堡工业大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；University Medical Center Hamburg-Eppendorf（汉堡-埃彭多夫大学医学中心）
Early detection of cancer is crucial for treatment and overall patient survival. In the upper aerodigestive tract (UADT) the gold standard for identification of malignant tissue is an invasive biopsy. Recently, non-invasive imaging techniques such as confocal laser microscopy and optical coherence tomography (OCT) have been used for tissue assessment. In particular, in a recent study experts classified lesions in the UADT with respect to their invasiveness using OCT images only. As the results were promising, automatic classification of lesions might be feasible which could assist experts in t…
Lesions are injuries and abnormal tissues in the human body. Detecting lesions in 3D Computed Tomography (CT) scans can be time-consuming even for very experienced physicians and radiologists. In recent years, CNN based lesion detectors have demonstrated huge potentials. Most of current state-of-the-art lesion detectors employ anchors to enumerate all possible bounding boxes with respect to the dataset in process. This anchor mechanism greatly improves the detection performance while also constraining the generalization ability of detectors. In this paper, we propose an anchor-free lesion dete…
University of Minho（米尼奥大学）；University Hospital Inselspital and University of Bern（Inselspital大学医院与伯尔尼大学）；University of Bern（伯尔尼大学）
Stroke is the second most common cause of death in developed countries, where rapid clinical intervention can have a major impact on a patient's life. To perform the revascularization procedure, the decision making of physicians considers its risks and benefits based on multi-modal MRI and clinical experience. Therefore, automatic prediction of the ischemic stroke lesion outcome has the potential to assist the physician towards a better stroke assessment and information about tissue outcome. Typically, automatic methods consider the information of the standard kinetic models of diffusion and p…
Universidad Técnica Particular de Loja（洛哈技术大学）；Universitat Politécnica de Valencia（瓦伦西亚理工大学）；University of Waterloo（滑铁卢大学）
Breast cancer is the most common invasive cancer in women, and the second main cause of death. Breast cancer screening is an efficient method to detect indeterminate breast lesions early. The common approaches of screening for women are tomosynthesis and mammography images. However, the traditional manual diagnosis requires an intense workload by pathologists, who are prone to diagnostic errors. Thus, the aim of this study is to build a deep convolutional neural network method for automatic detection, segmentation, and classification of breast lesions in mammography images. Based on deep learn…
Deep learning and classical computer vision techniques in medical image analysis: Case studies on brain MRI tissue segmentation, lung CT COPD registration, and skin lesion classification
University of Exeter（埃克塞特大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Sorbonne Université（索邦大学）
Department of Medicine I, LMU University Hospital, LMU Munich, Germany（慕尼黑大学医学部第一部门，LMU大学医院，慕尼黑，德国）；Lunit Inc.（Lunit公司）
Large annotated datasets are essential for training robust Computer-Aided Diagnosis (CAD) models for breast cancer detection or risk prediction. However, acquiring such datasets with fine-detailed annotation is both costly and time-consuming. Vision-Language Models (VLMs), such as CLIP, which are pre-trained on large image-text pairs, offer a promising solution by enhancing robustness and data efficiency in medical imaging tasks. This paper introduces a novel Multi-View Mammography and Language Model for breast cancer classification and risk prediction, trained on a dataset of paired mammogram…
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；Ping An Insurance Company of China（中国平安保险（集团）股份有限公司）
Automatic lesion detection from computed tomography (CT) scans is an important task in medical imaging analysis. It is still very challenging due to similar appearances (e.g. intensity and texture) between lesions and other tissues, making it especially difficult to develop a universal lesion detector. Instead of developing a specific-type lesion detector, this work builds a Universal Lesion Detector (ULDor) based on Mask R-CNN, which is able to detect all different kinds of lesions from whole body parts. As a state-of-the-art object detector, Mask R-CNN adds a branch for predicting segmentati…
Faculty of Medicine, American University of Beirut（贝鲁特美国大学医学院）；American University of Beirut Medical Center（贝鲁特美国大学医学中心）；Department of Diagnostic Radiology, American University of Beirut Medical Center（贝鲁特美国大学医学中心诊断放射科）
During their formative years, radiology trainees are required to interpret hundreds of mammograms per month, with the objective of becoming apt at discerning the subtle patterns differentiating benign from malignant lesions. Unfortunately, medico-legal and technical hurdles make it difficult to access and query medical images for training. In this paper we train a generative adversarial network (GAN) to synthesize 512 x 512 high-resolution mammograms. The resulting model leads to the unsupervised separation of high-level features (e.g. the standard mammography views and the nature of the breas…
Lesion-Aware Generative Artificial Intelligence for Virtual Contrast-Enhanced Mammography in Breast Cancer
Unit of Computer Systems and Bioinformatics, Department of Engineering, University Campus Bio-Medico of Rome（罗马大学生物医学校园计算机系统与生物信息学单位）；Department of Radiology, Fondazione Policlinico Campus Bio-Medico（生物医学校园放射科）；Department of Radiology, Università Campus Bio-Medico di Roma（罗马大学生物医学校园放射科）；Department of Radiation Sciences, Radiation Physics, Biomedical Engineering, Umeå University（乌梅大学辐射科学、辐射物理、生物医学工程部门）
Barcelona Supercomputing Center (BSC)（巴塞罗那超级计算中心）；Eidgenössische Technische Hochschule Zürich (ETH Zürich)（苏黎世联邦理工学院）；Universitat Politècnica de Catalunya (UPC)（加泰罗尼亚理工大学）
A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in order to diagnose hepatic diseases and assess the response to the according treatments. In this work we propose a method to segment the liver and its lesions from Computed Tomography (CT) scans using Convolutional Neural Networks (CNNs), that have proven good results in a variety of computer vision tasks, including medical imaging. The network that segments the lesions consists of a cascaded architecture, which first focuses on the region of the liver in order to segment the lesion…
Local Gamma Augmentation for Ischemic Stroke Lesion Segmentation on MRI
University of Copenhagen（哥本哈根大学）；Cerebriu A/S（Cerebriu公司）；Rigshospitalet, Copenhagen, Denmark（哥本哈根大学医院）；Pioneer Centre for Artificial Intelligence, University of Copenhagen（哥本哈根大学人工智能先锋中心）
Efficient Multi-scale Masked Autoencoders with Hybrid-Attention Mechanism for Breast Lesion Classification
Department of Electrical and Computer Engineering, University of Houston（电气与计算机工程系，德克萨斯大学）；Systems Medicine and Biomedical Engineering, Houston Methodist（系统医学与生物医学工程，休斯顿 Methodist）
Acute stroke lesion segmentation tasks are of great clinical interest as they can help doctors make better informed treatment decisions. Magnetic resonance imaging (MRI) is time demanding but can provide images that are considered gold standard for diagnosis. Automated stroke lesion segmentation can provide with an estimate of the location and volume of the lesioned tissue, which can help in the clinical practice to better assess and evaluate the risks of each treatment. We propose a deep learning methodology for acute and sub-acute stroke lesion segmentation using multimodal MR imaging. The p…
Mask-Guided Multi-Channel SwinUNETR Framework for Robust MRI Classification
Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain（巴塞罗那大学数学与计算机系）；Computer Vision Center, Bellaterra, Spain（贝尔特拉拉计算机视觉中心）；Institució Catalana de Recerca i Estudis Avançats (ICREA), Passeig Lluís Companys 23, Barcelona, Spain（加泰罗尼亚高级研究与教育研究所）
The Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院）；Sidney Kimmel Comprehensive Cancer Center（西德尼·基梅尔综合癌症中心）；The Johns Hopkins University（约翰霍普金斯大学）；University of Kentucky（肯塔基大学）；University of Wisconsin School of Medicine and Public Health（威斯康星大学医学与公共卫生学院）
A new paradigm is beginning to emerge in Radiology with the advent of increased computational capabilities and algorithms. This has led to the ability of real time learning by computer systems of different lesion types to help the radiologist in defining disease. For example, using a deep learning network, we developed and tested a multiparametric deep learning (MPDL) network for segmentation and classification using multiparametric magnetic resonance imaging (mpMRI) radiological images. The MPDL network was constructed from stacked sparse autoencoders with inputs from mpMRI. Evaluation of MPD…
A Lightweight CNN Integrated Compact Convolutional Transformer for Multi-Scale Feature Learning and reducing computational complexity for breast cancer mammography image detection and classification
A Hybrid Architecture for Benign-Malignant Classification of Mammography ROIs
Dept. of Electronics and Communication Engineering（电子与通信工程系）；Delhi Technological University（德里技术大学）；Dept. of Information Technology（信息科技系）；IGDTUW；Dept. of Computer Science and Engineering（计算机科学与工程系）
Modern Backbones Improve Multi-task DETR for Mammography Classification and Lesion Localization
University of Technology Sydney（悉尼科技大学）；Saigon Precision Medicine Research Center（西贡精准医学研究中心）；College of Medicine, Taipei Medical University（台北医学大学医学院）；Le Qui Don Technical University（黎文谍技术大学）；National Central University（中央大学）；AIBioMed Research Group, Taipei Medical University（台北医学大学AIBioMed研究组）
Interpretability in machine learning models is important in high-stakes decisions, such as whether to order a biopsy based on a mammographic exam. Mammography poses important challenges that are not present in other computer vision tasks: datasets are small, confounding information is present, and it can be difficult even for a radiologist to decide between watchful waiting and biopsy based on a mammogram alone. In this work, we present a framework for interpretable machine learning-based mammography. In addition to predicting whether a lesion is malignant or benign, our work aims to follow th…
School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院）；The Manuel and Raquel Klachky Chair of Image Processing, School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院 Manuel and Raquel Klachky 图像处理教授职位）；Diagnostic Imaging, Sheba Medical Center, affiliated to the Sackler School of Medicine, Tel-Aviv University（舍巴医疗中心诊断影像科，隶属于特拉维夫大学萨克勒医学院）
Mammography and ultrasound are extensively used by radiologists as complementary modalities to achieve better performance in breast cancer diagnosis. However, existing computer-aided diagnosis (CAD) systems for the breast are generally based on a single modality. In this work, we propose a deep-learning based method for classifying breast cancer lesions from their respective mammography and ultrasound images. We present various approaches and show a consistent improvement in performance when utilizing both modalities. The proposed approach is based on a GoogleNet architecture, fine-tuned for o…
Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation
Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院诊断放射学系）；Department of Pathology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院病理学系）；Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong（香港中文大学威尔斯亲王医院解剖及细胞病理学系）；Department of Infectious Diseases and Public Health, Jockey Club College of Veterinary Medicine and Life Sciences, City University of Hong Kong（香港城市大学赛马会动物医学及生命科学院传染病及公共卫生学系）；CityU Veterinary Diagnostic Laboratory Co., Ltd., City University of Hong Kong（香港城市大学城市大学兽医诊断实验室有限公司）
Standard breast cancer screening involves the acquisition of two mammography X-ray projections for each breast. Typically, a comparison of both views supports the challenging task of tumor detection and localization. We introduce a deep learning, patch-based Siamese network for lesion matching in dual-view mammography. Our locally-fitted approach generates a joint patch pair representation and comparison with a shared configuration between the two views. We performed a comprehensive set of experiments with the network on standard datasets, among them the large Digital Database for Screening Ma…
Data scarcity and class imbalance are two fundamental challenges in many machine learning applications to healthcare. Breast cancer classification in mammography exemplifies these challenges, with a malignancy rate of around 0.5% in a screening population, which is compounded by the relatively small size of lesions (~1% of the image) in malignant cases. Simultaneously, the prevalence of screening mammography creates a potential abundance of non-cancer exams to use for training. Altogether, these characteristics lead to overfitting on cancer cases, while under-utilizing non-cancer data. Here, w…
DynSegNet:Dynamic Architecture Adjustment for Adversarial Learning in Segmenting Hemorrhagic Lesions from Fundus Images
School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Information Engineering, Sichuan Agricultural University（四川农业大学信息工程学院）
School of Information Technology and Electrical Engineering, University of Queensland（昆士兰大学信息技术与电气工程学院）；Science and Engineering Faculty, Queensland University of Technology（昆士兰科技大学科学与工程学院）
Previous studies on computer aided detection/diagnosis (CAD) in 4D breast magnetic resonance imaging (MRI) regard lesion detection, segmentation and characterization as separate tasks, and typically require users to manually select 2D MRI slices or regions of interest as the input. In this work, we present a breast MRI CAD system that can handle 4D multimodal breast MRI data, and integrate lesion detection, segmentation and characterization with no user intervention. The proposed CAD system consists of three major stages: region candidate generation, feature extraction and region candidate cla…
Computer Vision and Robotics Institute, University of Girona（赫罗纳大学计算机视觉与机器人研究所）；University of Barcelona（巴塞罗那大学）；Faculty of Medicine, University of Girona（赫罗纳大学医学院）；Parc Tauli Hospital Universitari（陶利公园大学医院）；Institut d’Investigacio Parc Tauli I3PT（陶利公园研究所）
Medical image synthesis has gained a great focus recently, especially after the introduction of Generative Adversarial Networks (GANs). GANs have been used widely to provide anatomically-plausible and diverse samples for augmentation and other applications, including segmentation and super resolution. In our previous work, Deep Convolutional GANs were used to generate synthetic mammogram lesions, masses mainly, that could enhance the classification performance in imbalanced datasets. In this new work, a deeper investigation was carried out to explore other aspects of the generated images evalu…
A Dual-Task Synergy-Driven Generalization Framework for Pancreatic Cancer Segmentation in CT Scans
Southwest Jiaotong University（西南交通大学）；Shanghai Jiao Tong University（上海交通大学）；The Second Affiliated Hospital of Guangzhou Medical University（广州医科大学附属第二医院）
Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of Lausanne（洛桑大学）；Lausanne University Hospital（洛桑大学医院）；National Institute of Neurological Disorders and Stroke（美国国家神经疾病与卒中研究所）；National Institutes of Health（美国国立卫生研究院）；University of Bern（伯尔尼大学）；University of Pennsylvania（宾夕法尼亚大学）；Cedars-Sinai Medical Center（西达赛奈医疗中心）
The automated detection of cortical lesions (CLs) in patients with multiple sclerosis (MS) is a challenging task that, despite its clinical relevance, has received very little attention. Accurate detection of the small and scarce lesions requires specialized sequences and high or ultra-high field MRI. For supervised training based on multimodal structural MRI at 7T, two experts generated ground truth segmentation masks of 60 patients with 2014 CLs. We implemented a simplified 3D U-Net with three resolution levels (3D U-Net-). By increasing the complexity of the task (adding brain tissue segmen…
New York University（纽约大学）；Northwell Health（诺斯韦尔健康）
Applying deep learning methods to mammography assessment has remained a challenging topic. Dense noise with sparse expressions, mega-pixel raw data resolution, lack of diverse examples have all been factors affecting performance. The lack of pixel-level ground truths have especially limited segmentation methods in pushing beyond approximately bounding regions. We propose a classification approach grounded in high performance tissue assessment as an alternative to all-in-one localization and assessment models that is also capable of pinpointing the causal pixels. First, the objective of the mam…
This thesis focuses on the research and development of the Hemodynamic Tissue Signature (HTS) method: an unsupervised machine learning approach to describe the vascular heterogeneity of glioblastomas by means of perfusion MRI analysis. The HTS builds on the concept of habitats. An habitat is defined as a sub-region of the lesion with a particular MRI profile describing a specific physiological behavior. The HTS method delineates four habitats within the glioblastoma: the High Angiogenic Tumor (HAT) habitat, as the most perfused region of the enhancing tumor; the Low Angiogenic Tumor (LAT) habi…
NYU Center for Data Science（纽约大学数据科学中心）；Courant Institute of Mathematical Sciences, NYU（纽约大学库朗数学科学研究所）；NYU Grossman School of Medicine（纽约大学格罗斯曼医学院）；Jagiellonian University（雅盖隆大学）
In the last few years, deep learning classifiers have shown promising results in image-based medical diagnosis. However, interpreting the outputs of these models remains a challenge. In cancer diagnosis, interpretability can be achieved by localizing the region of the input image responsible for the output, i.e. the location of a lesion. Alternatively, segmentation or detection models can be trained with pixel-wise annotations indicating the locations of malignant lesions. Unfortunately, acquiring such labels is labor-intensive and requires medical expertise. To overcome this difficulty, weakl…
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究中心有限公司）
Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently invest…
Yildiz Technical University（耶尔德兹理工大学）；Istanbul Medipol University Hospital（伊斯坦布尔梅迪波尔大学医院）；Istanbul Technical University（伊斯坦布尔理工大学）
The cervical cancer developing from the precancerous lesions caused by the Human Papilloma Virus (HPV) has been one of the preventable cancers with the help of periodic screening. There are two types of grading conventions widely accepted among pathologists. On the other hand, inter-observer variability is an important issue for final diagnosis. In this paper, a whole-slide image grading benchmark for cervical cancer precursor lesions is introduced. The papillae of the cervical epithelium and overlapping cell problems are handled and a tissue classification method with a novel morphological fe…
Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and doses
Office of Science and Engineering Laboratories（科学与工程实验室办公室）；Center for Devices and Radiological Health（器械与放射卫生中心）；U.S. Food and Drug Administration（美国食品药品监督管理局）
A novel hierarchical model is introduced to solve a general problem of detecting groups of similar objects. Under this model, detection of groups is performed in hierarchically organized layers while each layer represents a scope for target objects. The processing of these layers involves sequential extraction of appearance features for an individual object, consistency measurement features for nearby objects, and finally the distribution features for all objects within the group. Using the concept of scope-based normalization, the extracted features not only enhance local contrast of an indiv…
Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, RadboudUMC Nijmegen（拉德堡德大学医学中心放射与核医学科诊断影像分析组）
We investigate the addition of symmetry and temporal context information to a deep Convolutional Neural Network (CNN) with the purpose of detecting malignant soft tissue lesions in mammography. We employ a simple linear mapping that takes the location of a mass candidate and maps it to either the contra-lateral or prior mammogram and Regions Of Interest (ROI) are extracted around each location. We subsequently explore two different architectures (1) a fusion model employing two datastreams were both ROIs are fed to the network during training and testing and (2) a stage-wise approach where a s…
