Accurate detection of all pathological findings in 3D medical images remains a significant challenge, as supervised models are limited to detecting only the few pathology classes annotated in existing datasets. To address this, we frame pathology detection as an unsupervised visual anomaly segmentation (UVAS) problem, leveraging the inherent rarity of pathological patterns compared to healthy ones. We enhance the existing density-based UVAS framework with two key innovations: (1) dense self-supervised learning for feature extraction, eliminating the need for supervised pretraining, and (2) learned, masking-invariant dense features as conditioning variables, replacing hand-crafted positional encodings. Trained on over 30,000 unlabeled 3D CT volumes, our fully self-supervised model, Screener, outperforms existing UVAS methods on four large-scale test datasets comprising 1,820 scans with diverse pathologies. Furthermore, in a supervised fine-tuning setting, Screener surpasses existing self-supervised pretraining methods, establishing it as a state-of-the-art foundation for pathology segmentation. The code and pretrained models will be made publicly available.
Technical University of Munich（慕尼黑工业大学）；Helmholtz Zentrum Munich（亥姆霍兹慕尼黑中心）；Imperial College London（帝国理工学院）
Detection of pathologies is a fundamental task in medical imaging and the evaluation of algorithms that can perform this task automatically is crucial. However, current object detection metrics for natural images do not reflect the specific clinical requirements in pathology detection sufficiently. To tackle this problem, we propose Robust Detection Outcome (RoDeO); a novel metric for evaluating algorithms for pathology detection in medical images, especially in chest X-rays. RoDeO evaluates different errors directly and individually, and reflects clinical needs better than current metrics. Ex…
MeDSLIP: Medical Dual-Stream Language-Image Pre-training with Pathology-Anatomy Semantic Alignment
Centre for Machine Intelligence and School of Computer Science, University of Sheffield（智能中心和计算机科学学院，谢菲尔德大学）；School of Medicine and Population Health, and INSIGNEO, Institute for in Silico Medicine, University of Sheffield（医学与人口健康学院，INSIGNEO，虚拟医学研究所，谢菲尔德大学）；Digital Environment Research Institute, Queen Mary University of London（数字环境研究所，伦敦大学玛丽女王学院）；School of Computer Science, University of Sheffield（计算机科学学院，谢菲尔德大学）；Department of Computing, Imperial College London（计算系，伦敦帝国学院）
Automatic Cardiac Pathology Recognition in Echocardiography Images Using Higher Order Dynamic Mode Decomposition and a Vision Transformer for Small Datasets
Universidad Politécnica de Madrid（马德里理工大学）；Centro Nacional de Investigaciones Cardiovasculares (CNIC)（国家心血管研究中心）；Universidad Complutense de Madrid（马德里康普顿斯大学）；Center for Computational Simulation (CCS)（计算仿真中心）
ERCPMP: An Endoscopic Image and Video Dataset for Colorectal Polyps Morphology and Pathology
Gastroenterology and Liver Disease Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences（沙希德贝赫什提医科大学胃肠病学与肝病研究所胃肠病学与肝病研究中心）；Alborz University of Medical Sciences（厄尔布尔士医科大学）；Shahid Beheshti University of Medical Sciences（沙希德贝赫什提医科大学）；Iran University of Medical Sciences（伊朗医科大学）
Supervoxel methods such as Simple Linear Iterative Clustering (SLIC) are an effective technique for partitioning an image or volume into locally similar regions, and are a common building block for the development of detection, segmentation and analysis methods. We introduce maskSLIC an extension of SLIC to create supervoxels within regions-of-interest, and demonstrate,on examples from 2-dimensions to 4-dimensions, that maskSLIC overcomes issues that affect SLIC within an irregular mask. We highlight the benefits of this method through examples, and show that it is able to better represent und…
Université de Sherbrooke（舍布鲁克大学）；Imagia cybernetics（Imagia cybernetics公司）；Google（谷歌）；Imeka Inc.（Imeka公司）
Segmentation of focal (localized) brain pathologies such as brain tumors and brain lesions caused by multiple sclerosis and ischemic strokes are necessary for medical diagnosis, surgical planning and disease development as well as other applications such as tractography. Over the years, attempts have been made to automate this process for both clinical and research reasons. In this regard, machine learning methods have long been a focus of attention. Over the past two years, the medical imaging field has seen a rise in the use of a particular branch of machine learning commonly known as deep l…
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）
In Digital Pathology (DP), labeled data is generally very scarce due to the requirement that medical experts provide annotations. We address this issue by learning transferable features from weakly labeled data, which are collected from various parts of the body and are organized by non-medical experts. In this paper, we show that features learned from such weakly labeled datasets are indeed transferable and allow us to achieve highly competitive patch classification results on the colorectal cancer (CRC) dataset [1] and the PatchCamelyon (PCam) dataset [2] while using an order of magnitude le…
Suboptimal generalization of machine learning models on unseen data is a key challenge which hampers the clinical applicability of such models to medical imaging. Although various methods such as domain adaptation and domain generalization have evolved to combat this challenge, learning robust and generalizable representations is core to medical image understanding, and continues to be a problem. Here, we propose STRAP (Style TRansfer Augmentation for histoPathology), a form of data augmentation based on random style transfer from non-medical style source such as artistic paintings, for learni…
Medical Image Synthesis via Fine-Grained Image-Text Alignment and Anatomy-Pathology Prompting
City University of Hong Kong（香港城市大学）；The Chinese University of Hong Kong（香港中文大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Lehigh University（里海大学）
针对医学图像数据稀缺及现有方法难以准确呈现解剖与病理细节的问题，提出结合解剖-病理提示和视觉码本细粒度图文对齐的合成模型，并在胸部 X 射线数据上验证其语义准确性。
IBM Research - Australia（IBM澳大利亚研究院）；EPF Laussane（洛桑联邦理工学院）
Anatomical landmark segmentation and pathology localization are important steps in automated analysis of medical images. They are particularly challenging when the anatomy or pathology is small, as in retinal images and cardiac MRI, or when the image is of low quality due to device acquisition parameters as in magnetic resonance (MR) scanners. We propose an image super-resolution method using progressive generative adversarial networks (P-GAN) that can take as input a low-resolution image and generate a high resolution image of desired scaling factor. The super resolved images can be used for …
Massachusetts General Hospital（麻省总医院）；Casey Eye Institute（凯西眼科研究所）；Brigham and Women’s Hospital（布莱根妇女医院）
Generative adversarial networks (GANs) are a class of unsupervised machine learning algorithms that can produce realistic images from randomly-sampled vectors in a multi-dimensional space. Until recently, it was not possible to generate realistic high-resolution images using GANs, which has limited their applicability to medical images that contain biomarkers only detectable at native resolution. Progressive growing of GANs is an approach wherein an image generator is trained to initially synthesize low resolution synthetic images (8x8 pixels), which are then fed to a discriminator that distin…
Medical datasets are often highly imbalanced with over-representation of common medical problems and a paucity of data from rare conditions. We propose simulation of pathology in images to overcome the above limitations. Using chest X-rays as a model medical image, we implement a generative adversarial network (GAN) to create artificial images based upon a modest sized labeled dataset. We employ a combination of real and artificial images to train a deep convolutional neural network (DCNN) to detect pathology across five classes of chest X-rays. Furthermore, we demonstrate that augmenting the …
Technical University of Munich (TUM)（慕尼黑工业大学）；Klinikum Rechts der Isar（伊萨尔河畔大学医院）；Imperial College London（伦敦帝国理工学院）
In medical imaging, un-, semi-, or self-supervised pathology detection is often approached with anomaly- or out-of-distribution detection methods, whose inductive biases are not intentionally directed towards detecting pathologies, and are therefore sub-optimal for this task. To tackle this problem, we propose AutoSeg, an engine that can generate diverse artificial anomalies that resemble the properties of real-world pathologies. Our method can accurately segment unseen artificial anomalies and outperforms existing methods for pathology detection on a challenging real-world dataset of Chest X-…
Linköping University（林雪平大学）；Center for Medical Image Science and Visualization, Linköping University（林雪平大学医学影像科学与可视化中心）；Sectra AB
Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of…
University of California San Diego（加利福尼亚大学圣地亚哥分校）；Tsinghua University（清华大学）；Wuhan University（武汉大学）；Beijing University of Technology（北京工业大学）；Carnegie Mellon University（卡内基梅隆大学）
Is it possible to develop an "AI Pathologist" to pass the board-certified examination of the American Board of Pathology (ABP)? To build such a system, three challenges need to be addressed. First, we need to create a visual question answering (VQA) dataset where the AI agent is presented with a pathology image together with a question and is asked to give the correct answer. Due to privacy concerns, pathology images are usually not publicly available. Besides, only well-trained pathologists can understand pathology images, but they barely have time to help create datasets for AI research. The…
The histological assessment of human tissue has emerged as the key challenge for detection and treatment of cancer. A plethora of different data sources ranging from tissue microarray data to gene expression, proteomics or metabolomics data provide a detailed overview of the health status of a patient. Medical doctors need to assess these information sources and they rely on data driven automatic analysis tools. Methods for classification, grouping and segmentation of heterogeneous data sources as well as regression of noisy dependencies and estimation of survival probabilities enter the proce…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medical College（威尔康奈尔医学院）
Before the recent success of deep learning methods for automated medical image analysis, practitioners used handcrafted radiomic features to quantitatively describe local patches of medical images. However, extracting discriminative radiomic features relies on accurate pathology localization, which is difficult to acquire in real-world settings. Despite advances in disease classification and localization from chest X-rays, many approaches fail to incorporate clinically-informed domain knowledge. For these reasons, we propose a Radiomics-Guided Transformer (RGT) that fuses \textit{global} image…
The University of Tokyo（东京大学）；National Center for Global Health and Medicine（国立国际医疗研究中心）；Kyushu Medical Center（九州医疗中心）
Convolutional Neural Networks (CNNs) can play a key role in Medical Image Analysis under large-scale annotated datasets. However, preparing such massive dataset is demanding. In this context, Generative Adversarial Networks (GANs) can generate realistic but novel samples, and thus effectively cover the real image distribution. In terms of interpolation, the GAN-based medical image augmentation is reliable because medical modalities can display the human body's strong anatomical consistency at fixed position while clearly reflecting inter-subject variability; thus, we propose to use noise-to-im…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Weill Cornell Medicine（威尔康奈尔医学院）
Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in image classification. We further point out that such imbalance fragility can be amplified when it comes to more sophisticated tasks such as pathology localization, as imbalances in such problems can have highly complex and often implicit forms of presence. For example, different pathology can have different sizes or colors (w.r.t.the background), different underlying demographic distributions, and in general different …
In-context learning enables multimodal large language models to classify cancer pathology images
National Center for Tumor Diseases (NCT)（国家肿瘤中心）；Heidelberg University Hospital（海德堡大学医院）；Else Kroener Fresenius Center for Digital Health（埃尔瑟·克勒纳·弗赖森纽斯数字健康中心）；Technical University Dresden（德累斯顿工业大学）；University of St Andrews（圣安德鲁斯大学）；Medical Faculty Mannheim, Heidelberg University（海德堡大学曼海姆医学院）；University Hospital Aachen（亚琛大学医院）；University Hospital Dresden（德累斯顿大学医院）
Microsoft Research（微软研究院）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Minnesota（明尼苏达大学）；Stanford University（斯坦福大学）；Duke University（杜克大学）；New York University（纽约大学）
Improving speed and image quality of Magnetic Resonance Imaging (MRI) via novel reconstruction approaches remains one of the highest impact applications for deep learning in medical imaging. The fastMRI dataset, unique in that it contains large volumes of raw MRI data, has enabled significant advances in accelerating MRI using deep learning-based reconstruction methods. While the impact of the fastMRI dataset on the field of medical imaging is unquestioned, the dataset currently lacks clinical expert pathology annotations, critical to addressing clinically relevant reconstruction frameworks an…
University of Dayton（代顿大学）；Deep Lens Inc（Deep Lens公司）
Deep Learning (DL) approaches have been providing state-of-the-art performance in different modalities in the field of medical imagining including Digital Pathology Image Analysis (DPIA). Out of many different DL approaches, Deep Convolutional Neural Network (DCNN) technique provides superior performance for classification, segmentation, and detection tasks. Most of the task in DPIA problems are somehow possible to solve with classification, segmentation, and detection approaches. In addition, sometimes pre and post-processing methods are applied for solving some specific type of problems. Rec…
PathReasoner-R1: Instilling Structured Reasoning into Pathology Vision-Language Model via Knowledge-Guided Policy Optimization
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Microsoft Research（微软研究院）；National University of Singapore（新加坡国立大学）；Technical University of Dresden（德累斯顿技术大学）
Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology
Institute of Pathology, Technical University of Munich（慕尼黑工业大学病理研究所）；Computer Aided Medical Procedures (CAMP), Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序（CAMP））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Human Pathology, Juntendo University Graduate School of Medicine（顺天堂大学医学研究生院人体病理学部）；The Hong Kong University of Science and Technology（香港科技大学）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）
SHAMSUL: Systematic Holistic Analysis to investigate Medical Significance Utilizing Local interpretability methods in deep learning for chest radiography pathology prediction
Stockholm University（斯德哥尔摩大学）；Skatturinn (Iceland Revenue and Customs)（冰岛税务总局（冰岛税收和海关局））
SpineBench: Benchmarking Multimodal LLMs for Spinal Pathology Analysis
Beijing University of Posts and Telecommunications（北京邮电大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Peking Union Medical College Hospital（北京佑安医学大学医院）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Institute for AI in Medicine, University Hospital Essen（埃森大学医院人工智能医学研究所）；University of Duisburg-Essen（杜伊斯堡-埃森大学）；German Cancer Consortium (DKTK)（德国癌症联盟）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；University Medicine Essen（埃森大学医学院）
When reading images, radiologists generate text reports describing the findings therein. Current state-of-the-art computer-aided diagnosis tools utilize a fixed set of predefined categories automatically extracted from these medical reports for training. This form of supervision limits the potential usage of models as they are unable to pick up on anomalies outside of their predefined set, thus, making it a necessity to retrain the classifier with additional data when faced with novel classes. In contrast, we investigate direct text supervision to break away from this closed set assumption. By…
Sorbonne Université（索邦大学）；Institut du Cerveau - Paris Brain Institute - ICM（巴黎大脑研究所（ICM））；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共援助医院）；Hôpital Pitié Salpêtrière（皮蒂耶-萨尔佩特里埃医院）
Non-invasive brain imaging techniques allow understanding the behavior and macro changes in the brain to determine the progress of a disease. However, computational pathology provides a deeper understanding of brain disorders at cellular level, able to consolidate a diagnosis and make the bridge between the medical image and the omics analysis. In traditional histopathology, histology slides are visually inspected, under the microscope, by trained pathologists. This process is time-consuming and labor-intensive; therefore, the emergence of Computational Pathology has triggered great hope to ea…
The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging
Technical University of Munich (TUM) and TUM University Hospital（慕尼黑工业大学和慕尼黑大学医院）；Technical University of Munich, School of Medicine and Health, TUM University Hospital, Klinikum rechts der Isar, Department of Diagnostic and Interventional Radiology（慕尼黑大学医学院和健康学院，慕尼黑大学医院，伊萨尔河右岸诊所，诊断与介入放射学系）；Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·阿列克桑德大学）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；Institute for Medical AI (IMAI), Heidelberg University Hospital and Medical Faculty of Heidelberg University（海德堡大学医院医学人工智能研究所和海德堡大学医学院）；Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology（德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯医学院 Else Kroener Fresenius 数字健康中心）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen (AöR)（埃森大学医院医学人工智能研究所）；Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen（埃森大学医院介入与诊断放射学及神经放射学研究所）；National Center for Tumor Diseases West（西德国立肿瘤疾病中心）
Johns Hopkins University（约翰斯·霍普金斯大学）；Sigma Technologies（12西格玛科技）；Johns Hopkins School of Medicine（约翰斯·霍普金斯医学院）
Medical images are often used to detect and characterize pathology and disease; however, automatically identifying and segmenting pathology in medical images is challenging because the appearance of pathology across diseases varies widely. To address this challenge, we propose a Bayesian deep learning method that learns to translate healthy computed tomography images to magnetic resonance images and simultaneously calculates voxel-wise uncertainty. Since high uncertainty occurs in pathological regions of the image, this uncertainty can be used for unsupervised anomaly segmentation. We show enc…
Fully supervised segmentation methods require a large training cohort of already segmented images, providing information at the pixel level of each image. We present a method to automatically segment and model pathologies in medical images, trained solely on data labelled on the image level as either healthy or containing a visual defect. We base our method on CycleGAN, an image-to-image translation technique, to translate images between the domains of healthy and pathological images. We extend the core idea with two key contributions. Implementing the generators as residual generators allows …
Adaptive Stain Normalization for Cross-Domain Medical Histology
Dept. of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA（生物医学工程系，约翰霍普金斯大学，巴尔的摩，MD，USA）；Johns Hopkins Malaria Research Institute, W. Harry Feinstone Department of Molecular Microbiology and Immunology, Bloomberg School of Public Health, Baltimore, MD, USA（约翰霍普金斯疟疾研究 institute，W. Harry Feinstone 分子微生物学与免疫学系，布隆伯格公共卫生学院，巴尔的摩，MD，USA）；Dept. of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA（医学系，约翰霍普金斯大学医学院，巴尔的摩，MD，USA）；Dept. of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA（电气与系统工程系，宾夕法尼亚大学，费城，PA，USA）
Emory University（埃默里大学）；Indiana University Purdue University-Indianapolis（印第安纳大学与普渡大学印第安纳波利斯联合分校）
Deep learning approaches applied to medical imaging have reached near-human or better-than-human performance on many diagnostic tasks. For instance, the CheXpert competition on detecting pathologies in chest x-rays has shown excellent multi-class classification performance. However, training and validating deep learning models require extensive collections of images and still produce false inferences, as identified by a human-in-the-loop. In this paper, we introduce a practical approach to improve the predictions of a pre-trained model through Few-Shot Learning (FSL). After training and valida…
