Mammographic breast density classification is essential for cancer risk assessment but remains challenging due to subjective interpretation and inter-observer variability. This study compares multimodal and CNN-based methods for automated classification using the BI-RADS system, evaluating BioMedCLIP and ConvNeXt across three learning scenarios: zero-shot classification, linear probing with textual descriptions, and fine-tuning with numerical labels. Results show that zero-shot classification achieved modest performance, while the fine-tuned ConvNeXt model outperformed the BioMedCLIP linear probe. Although linear probing demonstrated potential with pretrained embeddings, it was less effective than full fine-tuning. These findings suggest that despite the promise of multimodal learning, CNN-based models with end-to-end fine-tuning provide stronger performance for specialized medical imaging. The study underscores the need for more detailed textual representations and domain-specific adaptations in future radiology applications.
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital（马萨诸塞州总医院阿西诺拉·A·马丁诺斯生物医学成像中心）；Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）；University of Colorado School of Medicine（科罗拉多大学医学院）；Harvard Medical School（哈佛医学院）
Breast cancer is the most common cancers and early detection from mammography screening is crucial in improving patient outcomes. Assessing mammographic breast density is clinically important as the denser breasts have higher risk and are more likely to occlude tumors. Manual assessment by experts is both time-consuming and subject to inter-rater variability. As such, there has been increased interest in the development of deep learning methods for mammographic breast density assessment. Despite deep learning having demonstrated impressive performance in several prediction tasks for applicatio…
DeepHealth, Inc.（深健公司）；Rhode Island Hospital（罗德岛医院）；Brown University（布朗大学）；Henan Provincial People’s Hospital（河南省人民医院）；Medford Radiology Group（梅德福放射学集团）；University of Massachusetts Medical School（马萨诸塞大学医学院）
Breast cancer remains a global challenge, causing over 1 million deaths globally in 2018. To achieve earlier breast cancer detection, screening x-ray mammography is recommended by health organizations worldwide and has been estimated to decrease breast cancer mortality by 20-40%. Nevertheless, significant false positive and false negative rates, as well as high interpretation costs, leave opportunities for improving quality and access. To address these limitations, there has been much recent interest in applying deep learning to mammography; however, obtaining large amounts of annotated data p…
University of Edinburgh（爱丁堡大学）；Western General Hospital（西部综合医院）
Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis (CAD) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predict diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional considera…
BreastMammo and DenseMammo: Benchmarks for Mammography Domain Generalization
Northwestern University（西北大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Istanbul University（伊斯坦布尔大学）；Namik Kemal University（纳米克·凯末尔大学）；Istanbul Florence Nightingale Hospital（伊斯坦布尔佛罗伦萨南丁格尔医院）；Acibadem Mehmet Ali Aydinlar University（阿西巴德姆·穆罕默德·阿里·艾丁拉尔大学）；Qatar University（卡塔尔大学）；Boston University（波士顿大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；NVIDIA（英伟达公司）
Purpose: To develop a Breast Imaging Reporting and Data System (BI-RADS) breast density deep learning (DL) model in a multi-site setting for synthetic two-dimensional mammography (SM) images derived from digital breast tomosynthesis exams using full-field digital mammography (FFDM) images and limited SM data. Materials and Methods: A DL model was trained to predict BI-RADS breast density using FFDM images acquired from 2008 to 2017 (Site 1: 57492 patients, 187627 exams, 750752 images) for this retrospective study. The FFDM model was evaluated using SM datasets from two institutions (Site 1: 38…
Mammography-based screening has helped reduce the breast cancer mortality rate, but has also been associated with potential harms due to low specificity, leading to unnecessary exams or procedures, and low sensitivity. Digital breast tomosynthesis (DBT) improves on conventional mammography by increasing both sensitivity and specificity and is becoming common in clinical settings. However, deep learning (DL) models have been developed mainly on conventional 2D full-field digital mammography (FFDM) or scanned film images. Due to a lack of large annotated DBT datasets, it is difficult to train a …
Institute of Big Data（大数据研究所）；VinUniversity（文大大学）；Hospital 108（108医院）；Hanoi Medical University Hospital（河内医科大学医院）；Yale University（耶鲁大学）
Mammography, or breast X-ray, is the most widely used imaging modality to detect cancer and other breast diseases. Recent studies have shown that deep learning-based computer-assisted detection and diagnosis (CADe or CADx) tools have been developed to support physicians and improve the accuracy of interpreting mammography. However, most published datasets of mammography are either limited on sample size or digitalized from screen-film mammography (SFM), hindering the development of CADe and CADx tools which are developed based on full-field digital mammography (FFDM). To overcome this challeng…
A Multi-Modal AI System for Screening Mammography: Integrating 2D and 3D Imaging to Improve Breast Cancer Detection in a Prospective Clinical Study
Vilcek Institute of Graduate Biomedical Sciences, NYU Grossman School of Medicine（NYU Grossman医学院Vilcek研究生生物医学科学研究所）；NYU Grossman School of Medicine（NYU Grossman医学院）；Center for Data Science, New York University（纽约大学数据科学中心）；Emory University（埃默里大学）；Visage Imaging, Inc.（Visage成像公司）
Lovely Professional University（洛弗利专业大学）；Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Maharaja Agrasen Institute of Technology（马哈拉贾·阿格拉森技术学院）；University of Economics Ho Chi Minh City（胡志明市经济大学）
Breast cancer is a common cancer for women. Early detection of breast cancer can considerably increase the survival rate of women. This paper mainly focuses on transfer learning process to detect breast cancer. Modified VGG (MVGG), residual network, mobile network is proposed and implemented in this paper. DDSM dataset is used in this paper. Experimental results show that our proposed hybrid transfers learning model (Fusion of MVGG16 and ImageNet) provides an accuracy of 88.3% where the number of epoch is 15. On the other hand, only modified VGG 16 architecture (MVGG 16) provides an accuracy 8…
University of Edinburgh（爱丁堡大学）；Edinburgh Cancer Centre, Western General Hospital（西部综合医院爱丁堡癌症中心）
Computer-aided breast cancer diagnosis in mammography is limited by inadequate data and the similarity between benign and cancerous masses. To address this, we propose a signed graph regularized deep neural network with adversarial augmentation, named \textsc{DiagNet}. Firstly, we use adversarial learning to generate positive and negative mass-contained mammograms for each mass class. After that, a signed similarity graph is built upon the expanded data to further highlight the discrimination. Finally, a deep convolutional neural network is trained by jointly optimizing the signed graph regula…
Hangzhou Hikvision Digital Technology Co., Ltd.（杭州海康威视数字技术股份有限公司）
Background and Objective: Accurate detection of breast masses in mammography images is critical to diagnose early breast cancer, which can greatly improve the patients survival rate. However, it is still a big challenge due to the heterogeneity of breast masses and the complexity of their surrounding environment.Methods: To address these problems, we propose a one-stage object detection architecture, called Breast Mass Detection Network (BMassDNet), based on anchor-free and feature pyramid which makes the detection of breast masses of different sizes well adapted. We introduce a truncation nor…
Deep learning can promote the mammography-based computer-aided diagnosis (CAD) for breast cancers, but it generally suffers from the small sample size problem. Self-supervised learning (SSL) has shown its effectiveness in medical image analysis with limited training samples. However, the network model sometimes cannot be well pre-trained in the conventional SSL framework due to the limitation of the pretext task and fine-tuning mechanism. In this work, a Task-driven Self-supervised Bi-channel Networks (TSBN) framework is proposed to improve the performance of classification model the mammograp…
School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院）；The Manuel and Raquel Klachky Chair of Image Processing, School of Electrical Engineering, Tel-Aviv University（特拉维夫大学电气工程学院 Manuel and Raquel Klachky 图像处理教授职位）；Diagnostic Imaging, Sheba Medical Center, affiliated to the Sackler School of Medicine, Tel-Aviv University（舍巴医疗中心诊断影像科，隶属于特拉维夫大学萨克勒医学院）
Mammography and ultrasound are extensively used by radiologists as complementary modalities to achieve better performance in breast cancer diagnosis. However, existing computer-aided diagnosis (CAD) systems for the breast are generally based on a single modality. In this work, we propose a deep-learning based method for classifying breast cancer lesions from their respective mammography and ultrasound images. We present various approaches and show a consistent improvement in performance when utilizing both modalities. The proposed approach is based on a GoogleNet architecture, fine-tuned for o…
University of Girona（赫罗纳大学）；Parc Tauli Hospital Universitari（帕克陶利大学医院）；Institut d’Investigacio i Innovacio Parc Tauli (I3PT)（帕克陶利研究与创新研究所（I3PT））
Early detection of breast cancer has a major contribution to curability, and using mammographic images, this can be achieved non-invasively. Supervised deep learning, the dominant CADe tool currently, has played a great role in object detection in computer vision, but it suffers from a limiting property: the need of a large amount of labelled data. This becomes stricter when it comes to medical datasets which require high-cost and time-consuming annotations. Furthermore, medical datasets are usually imbalanced, a condition that often hinders classifiers performance. The aim of this paper is to…
Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Kaiser Permanente Northern California, Division of Research（北加州凯撒永久医疗集团研究部）
A major limitation in applying deep learning to artificial intelligence (AI) systems is the scarcity of high-quality curated datasets. We investigate strong augmentation based self-supervised learning (SSL) techniques to address this problem. Using breast cancer detection as an example, we first identify a mammogram-specific transformation paradigm and then systematically compare four recent SSL methods representing a diversity of approaches. We develop a method to convert a pretrained model from making predictions on uniformly tiled patches to whole images, and an attention-based pooling meth…
Breast cancer is in the most common malignant tumor in women. It accounted for 30% of new malignant tumor cases. Although the incidence of breast cancer remains high around the world, the mortality rate has been continuously reduced. This is mainly due to recent developments in molecular biology technology and improved level of comprehensive diagnosis and standard treatment. Early detection by mammography is an integral part of that. The most common breast abnormalities that may indicate breast cancer are masses and calcifications. Previous detection approaches usually obtain relatively high s…
Knowledge of x-ray attenuation is essential for developing and evaluating x-ray imaging technologies. For instance, techniques to distinguish between cysts and solid tumours at mammography screening would be highly desirable to reduce recalls, but the development requires knowledge of the x-ray attenuation for cysts and tumours. We have previously measured the attenuation of cyst fluid using photon-counting spectral mammography. Data on x-ray attenuation for solid breast lesions are available in the literature, but cover a relatively wide range, likely caused by natural spread between samples,…
Knowledge of x-ray attenuation is essential for developing and evaluating x-ray imaging technologies. For instance, techniques to better characterize cysts at mammography screening would be highly desirable to reduce recalls, but the development is hampered by the lack of attenuation data for cysts. We have developed a method to measure x-ray attenuation of tissue samples using a prototype photon-counting spectral mammography unit. The method was applied to measure the attenuation of 50 samples of breast cyst fluid and 50 samples of water. Spectral (energy-resolved) images of the samples were …
Beijing Institute of Technology（北京理工大学）；Université 8 Mai 1945 Guelma（盖勒马8 Mai 1945大学）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院肿瘤医院国家癌症中心国家肿瘤临床研究中心）
Mammography remains the most prevalent imaging tool for early breast cancer screening. The language used to describe abnormalities in mammographic reports is based on the breast Imaging Reporting and Data System (BI-RADS). Assigning a correct BI-RADS category to each examined mammogram is a strenuous and challenging task for even experts. This paper proposes a new and effective computer-aided diagnosis (CAD) system to classify mammographic masses into four assessment categories in BI-RADS. The mass regions are first enhanced by means of histogram equalization and then semiautomatically segment…
Computer-aided detection systems based on deep learning have shown great potential in breast cancer detection. However, the lack of domain generalization of artificial neural networks is an important obstacle to their deployment in changing clinical environments. In this work, we explore the domain generalization of deep learning methods for mass detection in digital mammography and analyze in-depth the sources of domain shift in a large-scale multi-center setting. To this end, we compare the performance of eight state-of-the-art detection methods, including Transformer-based models, trained i…
Reconsidering Explicit Longitudinal Mammography Alignment for Enhanced Breast Cancer Risk Prediction
Department of Physics and Technology, UiT The Arctic University of Norway（物理与技术系，UiT北极大学）；SPKI The Norwegian Centre for Clinical Artificial Intelligence（挪威临床人工智能中心）；Department of Computer Science and Engineering, University of Bologna（计算机科学与工程系，博洛尼亚大学）；Fraunhofer Research Institution for Individualized and Cell-Based Medical Engineering IMTE（个体化和细胞基础医学工程研究机构IMTE）；Norwegian Computing Center, Oslo, Norway（挪威计算中心，奥斯陆，挪威）；Pioneer Centre for AI, University of Copenhagen（人工智能先锋中心，哥本哈根大学）
Universidad Técnica Particular de Loja（洛哈技术大学）；Universitat Politécnica de Valencia（瓦伦西亚理工大学）；University of Waterloo（滑铁卢大学）
Breast cancer is the most common invasive cancer in women, and the second main cause of death. Breast cancer screening is an efficient method to detect indeterminate breast lesions early. The common approaches of screening for women are tomosynthesis and mammography images. However, the traditional manual diagnosis requires an intense workload by pathologists, who are prone to diagnostic errors. Thus, the aim of this study is to build a deep convolutional neural network method for automatic detection, segmentation, and classification of breast lesions in mammography images. Based on deep learn…
Transcend Research Consultancy（transcend研究咨询公司）；School of Computer Sciences, North Maharashtra University（北马哈拉施特拉大学计算机科学学院）
Mammography is a vital screening technique for early revealing and identification of breast cancer in order to assist to decrease mortality rate. Practical applications of mammograms are not limited to breast cancer revealing, identification ,but include task based lens design, image compression, image classification, content based image retrieval and a host of others. Mammography computational analysis methods are a useful tool for specialists to reveal hidden features and extract significant information in mammograms. Digital mammograms are mammography images available along with the convent…
Knowledge-based in silico models and dataset for the comparative evaluation of mammography AI for a range of breast characteristics, lesion conspicuities and doses
Office of Science and Engineering Laboratories（科学与工程实验室办公室）；Center for Devices and Radiological Health（器械与放射卫生中心）；U.S. Food and Drug Administration（美国食品药品监督管理局）
We work on the breast imaging malignancy segmentation task while focusing on the training process instead of network complexity. We designed a training process based on a modified U-Net, increasing the overall segmentation performances by using both, benign and malignant data for training. Our approach makes use of only a small amount of annotated data and relies on transfer learning from a self-supervised reconstruction task, and favors explainability.
Deep Learning-Based Breast Cancer Detection in Mammography: A Multi-Center Validation Study in Thai Population
Perceptra Co., Ltd（Perceptra公司）；Institute of Field Robotics（现场机器人研究所）；King Mongkut’s University of Technology Thonburi（技术师范大学）；Mahidol University（玛希多大学）；Siriraj Hospital（Siriraj医院）；Khonkaen University（孔卡恩大学）
Transformer-Based Explainable Deep Learning for Breast Cancer Detection in Mammography: The MammoFormer Framework
Morgan State University（摩根州立大学）；Wrexham University（雷克瑟姆大学）；Federal University of Technology, Minna（明纳联邦理工大学）；Federal University of Education, Kano（卡诺联邦教育大学）
Data scarcity and class imbalance are two fundamental challenges in many machine learning applications to healthcare. Breast cancer classification in mammography exemplifies these challenges, with a malignancy rate of around 0.5% in a screening population, which is compounded by the relatively small size of lesions (~1% of the image) in malignant cases. Simultaneously, the prevalence of screening mammography creates a potential abundance of non-cancer exams to use for training. Altogether, these characteristics lead to overfitting on cancer cases, while under-utilizing non-cancer data. Here, w…
Comparative Analysis of Segment Anything Model and U-Net for Breast Tumor Detection in Ultrasound and Mammography Images
Florida Atlantic University（佛罗里达大西洋大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）；Missouri University of Science and Technology（密苏里科技大学）；Florida International University（佛罗里达国际大学）
This work focuses on the automatic quantification of the breast density from digital mammography imaging. Using only categorical image-wise labels we train a model capable of predicting continuous density percentage as well as providing a pixel wise support frit for the dense region. In particular we propose a weakly supervised loss linking the density percentage to the mask size.
Image-Based Breast Implant Detection for Mammography Dataset Curation and Near-Real-Time Deployment: Comparing Foundation Models and Task-Specific Convolutional Models
Emory University School of Medicine（埃默里大学医学院）；University of Maryland（马里兰大学）
Early detection of breast cancer in mammography screening via deep-learning based computer-aided detection systems shows promising potential in improving the curability and mortality rates of breast cancer. However, many clinical centres are restricted in the amount and heterogeneity of available data to train such models to (i) achieve promising performance and to (ii) generalise well across acquisition protocols and domains. As sharing data between centres is restricted due to patient privacy concerns, we propose a potential solution: sharing trained generative models between centres as subs…
Early detection of breast cancer through screening mammography yields a 20-35% increase in survival rate; however, there are not enough radiologists to serve the growing population of women seeking screening mammography. Although commercial computer aided detection (CADe) software has been available to radiologists for decades, it has failed to improve the interpretation of full-field digital mammography (FFDM) images due to its low sensitivity over the spectrum of findings. In this work, we leverage a large set of FFDM images with loose bounding boxes of mammographically significant findings …
A major barrier to medical imaging research and in particular the development of artificial intelligence (AI) is a lack of large databases of medical images which share images with other researchers. Without such databases it is not possible to train generalisable AI algorithms, and large amounts of time and funding is spent collecting smaller datasets at individual research centres. The OPTIMAM image database (OMI-DB) has been developed to overcome these barriers. OMI-DB consists of several relational databases and cloud storage systems, containing mammography images and associated clinical a…
Center for Informatics, Federal University of Pernambuco（伯南布哥联邦大学信息学中心）；Department of Biomedical Engineering, Federal University of Pernambuco（伯南布哥联邦大学生物医学工程系）
Breast cancer is already one of the most common form of cancer worldwide. Mammography image analysis is still the most effective diagnostic method to promote the early detection of breast cancer. Accurately segmenting tumors in digital mammography images is important to improve diagnosis capabilities of health specialists and avoid misdiagnosis. In this work, we evaluate the feasibility of applying GrowCut to segment regions of tumor and we propose two GrowCut semi-supervised versions. All the analysis was performed by evaluating the application of segmentation techniques to a set of images ob…
Escola Politécnica, Universidade de São Paulo（圣保罗大学理工学院）；Instituto do Câncer do Estado de São Paulo（圣保罗州癌症研究所）；Faculdade de Medicina, Universidade de São Paulo（圣保罗大学医学院）
Some recent studies have described deep convolutional neural networks to diagnose breast cancer in mammograms with similar or even superior performance to that of human experts. One of the best techniques does two transfer learnings: the first uses a model trained on natural images to create a "patch classifier" that categorizes small subimages; the second uses the patch classifier to scan the whole mammogram and create the "single-view whole-image classifier". We propose to make a third transfer learning to obtain a "two-view classifier" to use the two mammographic views: bilateral craniocaud…
Mammographic Breast Positioning Assessment via Deep Learning
Istanbul Technical University（伊斯坦布尔理工大学）；Acibadem Healthcare Group（阿西巴德姆医疗集团）；Acibadem Mehmet Ali Aydinlar University（阿西巴德姆穆罕默德·阿里·艾丁拉尔大学）；Sisli Hamidiye Etfal Training and Research Hospital（锡什利哈米迪耶埃特法尔培训与研究医院）
Mayo Clinic（梅奥诊所）；Ohio State University（俄亥俄州立大学）；West Virginia University School of Medicine（西弗吉尼亚大学医学院）；NVIDIA Inc.（英伟达公司）
Screening mammograms are a routine imaging exam performed to detect breast cancer in its early stages to reduce morbidity and mortality attributed to this disease. In order to maximize the efficacy of breast cancer screening programs, proper mammographic positioning is paramount. Proper positioning ensures adequate visualization of breast tissue and is necessary for effective breast cancer detection. Therefore, breast-imaging radiologists must assess each mammogram for the adequacy of positioning before providing a final interpretation of the examination; this often necessitates return patient…
Institute for Systems and Robotics（系统与机器人研究所）；Interactive Technologies Institute（交互技术研究所）
This paper describes the field research, design and comparative deployment of a multimodal medical imaging user interface for breast screening. The main contributions described here are threefold: 1) The design of an advanced visual interface for multimodal diagnosis of breast cancer (BreastScreening); 2) Insights from the field comparison of single vs multimodality screening of breast cancer diagnosis with 31 clinicians and 566 images, and 3) The visualization of the two main types of breast lesions in the following image modalities: (i) MammoGraphy (MG) in both Craniocaudal (CC) and Mediolat…
Automatic identification of patients with luminal and non-luminal subtypes during a routine mammography screening can support clinicians in streamlining breast cancer therapy planning. Recent machine learning techniques have shown promising results in molecular subtype classification in mammography; however, they are highly dependent on pixel-level annotations, handcrafted, and radiomic features. In this work, we provide initial insights into the luminal subtype classification in full mammogram images trained using only image-level labels. Transfer learning is applied from a breast abnormality…
Fair Evaluation of Federated Learning Algorithms for Automated Breast Density Classification: The Results of the 2022 ACR-NCI-NVIDIA Federated Learning Challenge
American College of Radiology（美国放射学会）；The Massachusetts General Hospital（麻省总医院）；University of Colorado（科罗拉多大学）；National Institutes of Health National Cancer Institute（美国国立卫生研究院国家癌症研究所）；Computer Vision and Robotics Institute, University of Girona（赫罗纳大学计算机视觉与机器人研究所）；Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学合作Medianet创新中心）；Shanghai AI Laboratory（上海人工智能实验室）；Real Doctor AI Research Centre, Zhejiang University（浙江大学瑞医AI研究中心）；School of Public Health, Zhejiang University（浙江大学公共卫生学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；University of North Florida（北佛罗里达大学）
DSU-Net: An Attention-Enhanced Dense Skip U-Net for Breast Lesion Segmentation in Mammographic Images
Department of Biomedical Engineering, University of Wisconsin-Milwaukee（威斯康星大学密尔沃基分校生物医学工程系）；Department of Mechanical Engineering, Northern Arizona University（北亚利桑那大学机械工程系）
