Our study introduces ResNet-L2 (RL2), a novel metric for evaluating generative models and image quality in histopathology, addressing limitations of traditional metrics, such as Frechet inception distance (FID), when the data is scarce. RL2 leverages ResNet features with a normalizing flow to calculate RMSE distance in the latent space, providing reliable assessments across diverse histopathology datasets. We evaluated the performance of RL2 on degradation types, such as blur, Gaussian noise, salt-and-pepper noise, and rectangular patches, as well as diffusion processes. RL2's monotonic response to increasing degradation makes it well-suited for models that assess image quality, proving a valuable advancement for evaluating image generation techniques in histopathology. It can also be used to discard low-quality patches while sampling from a whole slide image. It is also significantly lighter and faster compared to traditional metrics and requires fewer images to give stable metric value.
University of Toronto（多伦多大学）；University of Alberta（阿尔伯塔大学）
Computational histopathology image diagnosis becomes increasingly popular and important, where images are segmented or classified for disease diagnosis by computers. While pathologists do not struggle with color variations in slides, computational solutions usually suffer from this critical issue. To address the issue of color variations in histopathology images, this study proposes two stain style transfer models, SSIM-GAN and DSCSI-GAN, based on the generative adversarial networks. By cooperating structural preservation metrics and feedback of an auxiliary diagnosis net in learning, medical-…
Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医疗中心）
We present an image translation approach to generate augmented data for mitigating data imbalances in a dataset of histopathology images of colorectal polyps, adenomatous tumors that can lead to colorectal cancer if left untreated. By applying cycle-consistent generative adversarial networks (CycleGANs) to a source domain of normal colonic mucosa images, we generate synthetic colorectal polyp images that belong to diagnostically less common polyp classes. Generated images maintain the general structure of their source image but exhibit adenomatous features that can be enhanced with our propose…
Generative Adversarial Networks for Stain Normalisation in Histopathology
University of Leeds（利兹大学）；Leeds Cancer Centre（利兹癌症中心）；St James’s University Hospital（圣詹姆斯大学医院）；Leeds Institute of Medical Research at St James’s（圣詹姆斯利兹医学研究所）；School of Medicine（医学院）；School of Computing（计算机学院）；Centre for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB)（生物医学计算成像与模拟技术中心（CISTIB））
University of Houston（休斯顿大学）；University Hospital of Cologne（科隆大学医院）；Weill Cornell Medicine（威尔康奈尔医学院）；Cornell University（康奈尔大学）；University of Turin（都灵大学）
Self-supervised learning (SSL) methods are enabling an increasing number of deep learning models to be trained on image datasets in domains where labels are difficult to obtain. These methods, however, struggle to scale to the high resolution of medical imaging datasets, where they are critical for achieving good generalization on label-scarce medical image datasets. In this work, we propose the Histopathology DatasetGAN (HDGAN) framework, an extension of the DatasetGAN semi-supervised framework for image generation and segmentation that scales well to large-resolution histopathology images. W…
University of Idaho（爱达荷大学）；Hunan University of Technology of Business（湖南工商大学）
Separating overlapped nuclei is a major challenge in histopathology image analysis. Recently published approaches have achieved promising overall performance on nuclei segmentation; however, their performance on separating overlapped nuclei is quite limited. To address the issue, we propose a novel multitask learning network with a bending loss regularizer to separate overlapped nuclei accurately. The newly proposed multitask learning architecture enhances the generalization by learning shared representation from three tasks: instance segmentation, nuclei distance map prediction, and overlappe…
We propose a virtual staining methodology based on Generative Adversarial Networks to map histopathology images of breast cancer tissue from H&E stain to PHH3 and vice versa. We use the resulting synthetic images to build Convolutional Neural Networks (CNN) for automatic detection of mitotic figures, a strong prognostic biomarker used in routine breast cancer diagnosis and grading. We propose several scenarios, in which CNN trained with synthetically generated histopathology images perform on par with or even better than the same baseline model trained with real images. We discuss the potentia…
University of British Columbia（不列颠哥伦比亚大学）；Eindhoven University of Technology（埃因霍温理工大学）；Massachusetts General Hospital（马萨诸塞州综合医院）；Harvard Medical School（哈佛医学院）；BC Cancer Agency（不列颠哥伦比亚癌症机构）
Visual microscopic study of diseased tissue by pathologists has been the cornerstone for cancer diagnosis and prognostication for more than a century. Recently, deep learning methods have made significant advances in the analysis and classification of tissue images. However, there has been limited work on the utility of such models in generating histopathology images. These synthetic images have several applications in pathology including utilities in education, proficiency testing, privacy, and data sharing. Recently, diffusion probabilistic models were introduced to generate high quality ima…
A Generative Foundation Model for Multimodal Histopathology
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤学系）；Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine（斯坦福大学医学院精神病学与行为科学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Radiology, Stanford University（斯坦福大学放射学系）；Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）
Retrieval-Guided Generation for Safer Histopathology Image Captioning
Kimia Lab, Department of Artificial Intelligence \& Informatics, Mayo Clinic, Rochester, MN, USA；Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA；Lane Department of Computer Science；Electrical Engineering, West Virginia University, Morgantown, WV, USA；Department of Computer Science；Engineering, Princeton University, Princeton, NJ, USA；Department of Computer Sciences, University of Wisconsin--Madison, Madison, WI, USA
F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
University of Utah（犹他大学）；The Catholic University of Korea College of Medicine（韩国天主教大学医学院）；Huntsman Cancer Institute（亨茨曼癌症研究所）；Scientific Computing and Imaging Institute（科学计算与成像研究所）
Due to the recent advancements in machine vision, digital pathology has gained significant attention. Histopathology images are distinctly rich in visual information. The tissue glass slide images are utilized for disease diagnosis. Researchers study many methods to process histopathology images and facilitate fast and reliable diagnosis; therefore, the availability of high-quality slides becomes paramount. The quality of the images can be negatively affected when the glass slides are ink-marked by pathologists to delineate regions of interest. As an example, in one of the largest public histo…
The distribution and appearance of nuclei are essential markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology, there is a lack of large scale, accurate, publicly accessible nucleus segmentation data. To address this, we developed an analysis pipeline that segments nuclei in whole slide tissue images from multiple cancer types with a quality control process. We have generated nucleus segmentation results in 5,060 Whole Slide Tissue images from 10 cancer types in The Cancer Genome Atlas. One key component of our work is that we carried out a multi-level qual…
Technion - IIT（以色列理工学院）；Rambam Health Care Campus（兰巴姆医疗中心）
Artificial intelligence and machine learning techniques have the promise to revolutionize the field of digital pathology. However, these models demand considerable amounts of data, while the availability of unbiased training data is limited. Synthetic images can augment existing datasets, to improve and validate AI algorithms. Yet, controlling the exact distribution of cellular features within them is still challenging. One of the solutions is harnessing conditional generative adversarial networks that take a semantic mask as an input rather than a random noise. Unlike other domains, outlining…
Pennsylvania State University（宾夕法尼亚州立大学）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
Cervical intraepithelial neoplasia (CIN) grade of histopathology images is a crucial indicator in cervical biopsy results. Accurate CIN grading of epithelium regions helps pathologists with precancerous lesion diagnosis and treatment planning. Although an automated CIN grading system has been desired, supervised training of such a system would require a large amount of expert annotations, which are expensive and time-consuming to collect. In this paper, we investigate the CIN grade classification problem on segmented epithelium patches. We propose to use conditional Generative Adversarial Netw…
University of Toronto（多伦多大学）；University of British Columbia（不列颠哥伦比亚大学）；Sunnybrook Research Institute（森尼布鲁克研究所）
Unsupervised learning has been a long-standing goal of machine learning and is especially important for medical image analysis, where the learning can compensate for the scarcity of labeled datasets. A promising subclass of unsupervised learning is self-supervised learning, which aims to learn salient features using the raw input as the learning signal. In this paper, we use a contrastive self-supervised learning method called SimCLR that achieved state-of-the-art results on natural-scene images and apply this method to digital histopathology by collecting and pretraining on 57 histopathology …
There is a strong need for automated systems to improve diagnostic quality and reduce the analysis time in histopathology image processing. Automated detection and classification of pathological tissue characteristics with computer-aided diagnostic systems are a critical step in the early diagnosis and treatment of diseases. Once a pathology image is scanned by a microscope and loaded onto a computer, it can be used for automated detection and classification of diseases. In this study, the DenseNet-161 and ResNet-50 pre-trained CNN models have been used to classify digital histopathology patch…
Nuclei segmentation is a fundamental task that is critical for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Conventional vision-based methods for nuclei segmentation struggle in challenging cases and deep learning approaches have proven to be more robust and generalizable. However, CNNs require large amounts of labeled histopathology data. Moreover, conventional CNN-based approaches lack structured prediction capabilities which are required to distinguish overlapping and clumped nuclei. Here, we present an appr…
University Joseph Fourier（约瑟夫·傅里叶大学）；IPAL, CNRS UMI 2955（IPAL，法国国家科学研究中心UMI 2955）；CoSMo Software（CoSMo软件公司）；Temasys Communications（Temasys通信公司）；University Pierre and Marie Curie（皮埃尔和玛丽·居里大学）
Accurate detection of mitosis plays a critical role in breast cancer histopathology. Manual detection and counting of mitosis is tedious and subject to considerable inter- and intra-reader variations. Multispectral imaging is a recent medical imaging technology, proven successful in increasing the segmentation accuracy in other fields. This study aims at improving the accuracy of mitosis detection by developing a specific solution using multispectral and multifocal imaging of breast cancer histopathological data. We propose to enable clinical routine-compliant quality of mitosis discrimination…
DiffInfinite: Large Mask-Image Synthesis via Parallel Random Patch Diffusion in Histopathology
University of Glasgow（格拉斯哥大学）；Dotphoton；Fraunhofer HHI（弗劳恩霍夫海因里希·赫兹研究所）；Aignostics；Institute of Pathology, LMU Munich（慕尼黑大学病理学研究所）；UC Berkeley（加州大学伯克利分校）；TU Berlin（柏林工业大学）
Generating crossmodal gene expression from cancer histopathology improves multimodal AI predictions
School of Mathematical & Computational Sciences, Indian Association for the Cultivation of Science（数学与计算科学学院，印度科学培养协会）；The Alan Turing Institute（艾伦·图灵研究所）；Comprehensive Cancer Center, King’s College London（国王学院综合癌症中心）；Department of Computer Science and Engineering, Jadavpur University（计算机科学与工程系，贾瓦德pur大学）；MRC Biostatistics Unit, University of Cambridge（剑桥大学医学研究委员会生物统计学单位）；Department of Biostatistics, Bioinformatics and Biomathematics, Georgetown University（生物统计学、生物信息学与生物数学系，杰斐逊大学）；UCL Cancer Institute, Dept of Medical Physics & Biomedical Engineering, University College London（伦敦大学学院癌症研究所，医学物理与生物医学工程系）
The Pennsylvania State University（宾夕法尼亚州立大学）；Lister Hill National Center for Biomedical Communications（利斯特希尔国家生物医学通信中心）；National Library of Medicine（国家医学图书馆）；Penn State Health Milton S. Hershey Medical Center（宾夕法尼亚州立大学健康米尔顿·S·赫尔希医学中心）；Penn State College Of Medicine（宾夕法尼亚州立大学医学院）
Histopathological analysis is the present gold standard for precancerous lesion diagnosis. The goal of automated histopathological classification from digital images requires supervised training, which requires a large number of expert annotations that can be expensive and time-consuming to collect. Meanwhile, accurate classification of image patches cropped from whole-slide images is essential for standard sliding window based histopathology slide classification methods. To mitigate these issues, we propose a carefully designed conditional GAN model, namely HistoGAN, for synthesizing realisti…
A Chain-of-thought Reasoning Breast Ultrasound Dataset Covering All Histopathology Categories
Peking University（北京大学）；Peking Union Medical College Hospital（北京大学人民医院）；Peking University Cancer Hospital & Institute（北京大学肿瘤医院）；National Cancer Center / National Clinical Research Center for Cancer / Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院国家癌症中心/国家癌症临床科研中心/癌症医院）；Shenzhen Maternity & Child Health Care Hospital（深圳妇幼保健医院）；Xi’an Jiaotong University（西安交通大学）；Yizhun Medical AI Co., Ltd（亿众医疗人工智能有限公司）
HARE: an entity and relation centric evaluation framework for histopathology reports
University College London（伦敦大学学院）；University of Glasgow（格拉斯哥大学）；Human Technopole（人类技术极地）；University College London Hospitals NHS Foundation Trust（伦敦大学学院医院国家健康服务信托）
本文提出以实体和关系为中心的组织病理学报告评估框架 HARE，通过构建标注基准、微调 NER 与 RE 模型并设计新指标，在专家评估相关性上优于传统及放射学指标。
Faculty of Informatics and Data Science, University of Regensburg, Germany（信息学与数据科学学院，雷根萨大学）；Indian Institute of Information Technology Bhopal, India（比哈尔信息学院）
Comparative Analysis of Diffusion Generative Models in Computational Pathology
Concordia University（康考迪亚大学）；University of Montreal Hospital Center（蒙特利尔大学医院中心）；Kingston Health Sciences Center（金斯顿健康科学中心）；University of Calgary（卡尔加里大学）；Mila–Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
Learning from the Right Patches: A Two-Stage Wavelet-Driven Masked Autoencoder for Histopathology Representation Learning
PLRI Medical Informatics Institute（PLRI医学信息学研究所）；CAIMed Reserch Center（CAIMed研究中心）；Hannover Medical School（汉诺威医学院）；Computer Science Institute（计算机科学研究所）；Leibniz University Hannover（莱比锡大学汉诺威分校）；Sanford Burnham Prebys Medical Discovery Institute（桑福医疗发现研究所）；Rady Children’s Institute for Genomic Medicine（拉迪儿童基因医学研究所）；University of California San Diego（加州大学圣地亚哥分校）
HistDiT: A Structure-Aware Latent Conditional Diffusion Model for High-Fidelity Virtual Staining in Histopathology
Edge Hill University（埃奇希尔大学）；University of Nottingham Malaysia（诺丁汉大学马来西亚分校）；University of Southampton Malaysia（南安普顿大学马来西亚分校）；Cancer Research Malaysia（马来西亚癌症研究中心）
Google AI Healthcare（谷歌AI医疗保健）；Avoneaux Medical Institute（阿沃诺医疗研究所）；University of Tennessee Health Science Center（田纳西大学健康科学中心）
The increasing availability of large institutional and public histopathology image datasets is enabling the searching of these datasets for diagnosis, research, and education. Though these datasets typically have associated metadata such as diagnosis or clinical notes, even carefully curated datasets rarely contain annotations of the location of regions of interest on each image. Because pathology images are extremely large (up to 100,000 pixels in each dimension), further laborious visual search of each image may be needed to find the feature of interest. In this paper, we introduce a deep le…
Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have…
SYNTA: A novel approach for deep learning-based image analysis in muscle histopathology using photo-realistic synthetic data
MIRA Vision Microscopy GmbH（MIRA Vision 显微镜有限公司）；Friedrich-Alexander University Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Universitätsklinikum Erlangen（埃尔朗根大学医院）；German Aerospace Center (DLR)（德国航空航天中心）；University of Cologne（科隆大学）；Leibniz-Institut für Analytische Wissenschaften - ISAS - e.V.（莱布尼茨分析科学研究所）
本文提出 SYNTA 方法，通过生成照片级逼真的合成生物医学图像作为深度学习训练数据，在无需人工标注的情况下实现了对真实肌肉组织切片的专家级分割，为 GAN 提供了可解释且可控的替代方案。
Large annotated datasets have been a key component in the success of deep learning. However, annotating medical images is challenging as it requires expertise and a large budget. In particular, annotating different types of cells in histopathology suffer from high inter- and intra-rater variability due to the ambiguity of the task. Under this setting, the relation between annotators' variability and model performance has received little attention. We present a large-scale study on the variability of cell annotations among 120 board-certified pathologists and how it affects the performance of a…
Current medical image synthetic augmentation techniques rely on intensive use of generative adversarial networks (GANs). However, the nature of GAN architecture leads to heavy computational resources to produce synthetic images and the augmentation process requires multiple stages to complete. To address these challenges, we introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. The teacher learns to generate curriculum to feed into the student model for data augmentation and guides the stude…
Center for Wireless Networks & Applications (WNA), Amrita Vishwa Vidyapeetham（阿姆里塔·维什瓦·维迪亚皮塔姆大学无线网络与应用中心）；Valeo（法雷奥）；Department of Computer Science and Engineering, Amrita Vishwa Vidyapeetham（阿姆里塔·维什瓦·维迪亚皮塔姆大学计算机科学与工程系）；School of Computing and Electrical Engineering, IIT Mandi（印度理工学院曼迪分校计算与电气工程学院）
Computer assisted diagnosis in digital pathology is becoming ubiquitous as it can provide more efficient and objective healthcare diagnostics. Recent advances have shown that the convolutional Neural Network (CNN) architectures, a well-established deep learning paradigm, can be used to design a Computer Aided Diagnostic (CAD) System for breast cancer detection. However, the challenges due to stain variability and the effect of stain normalization with such deep learning frameworks are yet to be well explored. Moreover, performance analysis with arguably more efficient network models, which may…
Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation
Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院诊断放射学系）；Department of Pathology, Li Ka Shing Faculty of Medicine, The University of Hong Kong（香港大学李嘉诚医学院病理学系）；Department of Anatomical and Cellular Pathology, Prince of Wales Hospital, The Chinese University of Hong Kong（香港中文大学威尔斯亲王医院解剖及细胞病理学系）；Department of Infectious Diseases and Public Health, Jockey Club College of Veterinary Medicine and Life Sciences, City University of Hong Kong（香港城市大学赛马会动物医学及生命科学院传染病及公共卫生学系）；CityU Veterinary Diagnostic Laboratory Co., Ltd., City University of Hong Kong（香港城市大学城市大学兽医诊断实验室有限公司）
