Background. Digital pathology has aroused widespread interest in modern pathology. The key of digitalization is to scan the whole slide image (WSI) at high magnification. The lager the magnification is, the richer details WSI will provide, but the scanning time is longer and the file size of obtained is larger. Methods. We design a strategy to scan slides with low resolution (5X) and a super-resolution method is proposed to restore the image details when in diagnosis. The method is based on a multi-scale generative adversarial network, which sequentially generates three high-resolution images such as 10X, 20X and 40X. Results. The peak-signal-to-noise-ratio of 10X to 40X generated images are 24.16, 22.27 and 20.44, and the structural-similarity-index are 0.845, 0.680 and 0.512, which are better than other super-resolution networks. Visual scoring average and standard deviation from three pathologists is 3.63 plus-minus 0.52, 3.70 plus-minus 0.57 and 3.74 plus-minus 0.56 and the p value of analysis of variance is 0.37, indicating that generated images include sufficient information for diagnosis. The average value of Kappa test is 0.99, meaning the diagnosis of generated images is highly consistent with that of the real images. Conclusion. This proposed method can generate high-quality 10X, 20X, 40X images from 5X images at the same time, in which the time and storage costs of digitalization can be effectively reduced up to 1/64 of the previous costs. The proposed method provides a better alternative for low-cost storage, faster image share of digital pathology. Keywords. Digital pathology; Super-resolution; Low resolution scanning; Low cost
UWarp: A Whole Slide Image Registration Pipeline to Characterize Scanner-Induced Local Domain Shift
Ummon HealthTech SAS；Centre Georges-François Leclerc（乔治-弗朗索瓦·勒克莱尔中心）；ICMUB laboratory, Université de Bourgogne Europe（勃艮第欧洲大学ICMUB实验室）；University Hospital of Dijon（第戎大学医院）；Hospital de Tortosa Verge de la Cinta (HTVC), Catalan Health Institute (ICS)（托尔托萨维尔赫-德拉辛塔医院 (HTVC)，加泰罗尼亚卫生研究所 (ICS)）；Pere Virgili Institute for Health Research (IISPV)（佩雷·维尔吉利健康研究所 (IISPV)）；Universitat Rovira i Virgili (URV)（罗维拉-威尔吉利大学 (URV)）；Nałęcz Institute of Biocybernetics and Biomedical Engineering Polish Academy of Sciences（波兰科学院纳文奇生物控制论与生物医学工程研究所）；Cairo University（开罗大学）；University of Udine（乌迪内大学）
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Massachusetts Institute of Technology（麻省理工学院）
Optical coherence tomography (OCT) is a micrometer-scale, volumetric imaging modality that has become a clinical standard in ophthalmology. OCT instruments image by raster-scanning a focused light spot across the retina, acquiring sequential cross-sectional images to generate volumetric data. Patient eye motion during the acquisition poses unique challenges: Non-rigid, discontinuous distortions can occur, leading to gaps in data and distorted topographic measurements. We present a new distortion model and a corresponding fully-automatic, reference-free optimization strategy for computational m…
A Spatial-Physics Informed Model for 3D Spiral Sample Scanned by SQUID Microscopy
Institute for Infocomm Research (I2R), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局信息通信研究院）；National University of Singapore（新加坡国立大学）；Neocera Magma；University of Maryland（马里兰大学）
This work presents a new cyclic architecture that extracts high-frequency patterns from images and re-insert them as geometric features. This procedure allows us to enhance the resolution of low-cost depth sensors capturing fine details on the one hand and being loyal to the scanned ground truth on the other. We present state-of-the-art results for depth super-resolution tasks and as well as visually attractive, enhanced generated 3D models.
University of Bonn（波恩大学）；FZ Jülich GmbH（于利希研究中心有限公司）
Magnetic resonance imaging (MRI) enables plant scientists to non-invasively study root system development and root-soil interaction. Challenging recording conditions, such as low resolution and a high level of noise hamper the performance of traditional root extraction algorithms, though. We propose to increase signal-to-noise ratio and resolution by segmenting the scanned volumes into root and soil in super-resolution using a 3D U-Net. Tests on real data show that the trained network is capable to detect most roots successfully and even finds roots that were missed by human annotators. Our ex…
Super-resolution microscopes (such as STED) illuminate samples with a tiny spot, and achieve very high resolution. But structures smaller than the spot cannot be resolved in this way. Therefore, we propose a technique to solve this problem. It is termed "Deconvolution after Dense Scan (DDS)". First, a preprocessing stage is introduced to eliminate the optical uncertainty of the peripheral areas around the sample's ROI (Region of Interest). Then, the ROI is scanned densely together with its peripheral areas. Finally, the high resolution image is recovered by deconvolution. The proposed techniqu…
Robustness and Diagnostic Performance of Super-Resolution Fetal Brain MRI
Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学医学学院）；University Medical Centre Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学医学中心）；Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia（卢布尔雅那大学电气工程学院）
Department of Computer Science, Vanderbilt University（范德堡大学计算机科学系）
Histopathology has played an essential role in cancer diagnosis. With the rapid advances in convolutional neural networks (CNN). Various CNN-based automated pathological image segmentation approaches have been developed in computer-assisted pathological image analysis. In the past few years, Transformer neural networks (Transformer) have shown the unique merit of capturing the global long-distance dependencies across the entire image as a new deep learning paradigm. Such merit is appealing for exploring spatially heterogeneous pathological images. However, there have been very few, if any, stu…
A Frequency-Aware Self-Supervised Learning for Ultra-Wide-Field Image Enhancement
College of Information Engineering, Zhejiang University of Technology, Hangzhou, China（浙江工业大学信息工程学院）；Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, China（中国科学院宁波材料技术与工程研究所）；Department of Eye and Vision Science, University of Liverpool, Liverpool, UK（利物浦大学眼科学与视觉科学系）；School of Biomedical Engineering, ShanghaiTech University, Shanghai, China（上海科技大学生物医学工程学院）
A Self-Supervised Image Registration Approach for Measuring Local Response Patterns in Metastatic Ovarian Cancer
University of Cambridge（剑桥大学）；Early Cancer Institute（早期癌症研究所）；Technical University of Munich（慕尼黑工业大学）；Helmholtz Munich（慕尼黑亥姆霍兹中心）；University of Salerno（萨勒诺大学）
Vanderbilt University（范德堡大学）；Big Data Institute, Central South University（中南大学大数据研究院）；Vanderbilt University Medical Center（范德堡大学医学中心）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Comprehensive semantic segmentation on renal pathological images is challenging due to the heterogeneous scales of the objects. For example, on a whole slide image (WSI), the cross-sectional areas of glomeruli can be 64 times larger than that of the peritubular capillaries, making it impractical to segment both objects on the same patch, at the same scale. To handle this scaling issue, prior studies have typically trained multiple segmentation networks in order to match the optimal pixel resolution of heterogeneous tissue types. This multi-network solution is resource-intensive and fails to mo…
Whole Slide Concepts: A Supervised Foundation Model For Pathological Images
Institute for Digital Medicine MEVIS（数字医学MEVIS研究所）；Fraunhofer（弗劳恩霍夫）；Institute for Toxicology and Experimental Medicine（毒理学与实验医学研究所）；Institute for Diagnostic and Interventional Radiology（诊断与介入放射学研究所）；Hannover Medical School（汉诺威医学院）
Central South University（中南大学）；Shaanxi Provincial People’s Hospital（陕西省人民医院）；Xiangya Hospital, Central South University（中南大学湘雅医院）
[Purpose] The pathology is decisive for disease diagnosis, but relies heavily on the experienced pathologists. Recently, pathological artificial intelligence (PAI) is thought to improve diagnostic accuracy and efficiency. However, the high performance of PAI based on deep learning in the laboratory generally cannot be reproduced in the clinic. [Methods] Because the data preparation is important for PAI, the paper has reviewed PAI-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. The in-depth analysis of methods for preparing dat…
Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；School of Science, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）理学院）
Classification of lung cancer subtypes on CT images with synthetic pathological priors
Research Center for Healthcare Data Science（医疗数据科学研究中心）；Zhejiang Lab（浙江大学实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；Northwestern Polytechnical University（西北工业大学）；Institute of Computer Graphics and Vision（计算机图形学与视觉研究所）；Graz University of Technology（格拉茨工业大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of Computer Science（计算机科学系）；Rutgers University（罗格斯大学）
PAII Inc.（PAII公司）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Chang Gung Memorial Hospital（长庚纪念医院）；PingAn Technology（平安科技）
In medical imaging, organ/pathology segmentation models trained on current publicly available and fully-annotated datasets usually do not well-represent the heterogeneous modalities, phases, pathologies, and clinical scenarios encountered in real environments. On the other hand, there are tremendous amounts of unlabelled patient imaging scans stored by many modern clinical centers. In this work, we present a novel segmentation strategy, co-heterogenous and adaptive segmentation (CHASe), which only requires a small labeled cohort of single phase imaging data to adapt to any unlabeled cohort of …
Uncovering Latent Pathological Signatures in Pulmonary CT via Cross-Window Knowledge Distillation
Institute of Health Informatics, University College London（伦敦大学学院健康信息学研究所）；Department of Pulmonary and Critical Care Medicine, Shanghai East Hospital, School of Medicine, Tongji University（同济大学医学院 pulmonary and critical care medicine 部门，上海东方医院）；Queen Mary University of London（伦敦女王玛丽大学）；School of Health and Wellbeing, University of Glasgow（格拉斯哥大学健康与福祉学院）
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences（广东省人民医院（广东省医学科学院））；Southern Medical University（南方医科大学）
Color inconsistency is an inevitable challenge in computational pathology, which generally happens because of stain intensity variations or sections scanned by different scanners. It harms the pathological image analysis methods, especially the learning-based models. A series of approaches have been proposed for stain normalization. However, most of them are lack flexibility in practice. In this paper, we formulated stain normalization as a digital re-staining process and proposed a self-supervised learning model, which is called RestainNet. Our network is regarded as a digital restainer which…
PAII Inc.（PAII公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；Vanderbilt University（范德堡大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Multi-modal image registration is a challenging problem that is also an important clinical task for many real applications and scenarios. As a first step in analysis, deformable registration among different image modalities is often required in order to provide complementary visual information. During registration, semantic information is key to match homologous points and pixels. Nevertheless, many conventional registration methods are incapable in capturing high-level semantic anatomical dense correspondences. In this work, we propose a novel multi-task learning system, JSSR, based on an end…
Northeastern University（东北大学）；University of the Chinese Academy of Sciences（中国科学院大学）；China Medical University（中国医科大学）
The deep neural network is a research hotspot for histopathological image analysis, which can improve the efficiency and accuracy of diagnosis for pathologists or be used for disease screening. The whole slide pathological image can reach one gigapixel and contains abundant tissue feature information, which needs to be divided into a lot of patches in the training and inference stages. This will lead to a long convergence time and large memory consumption. Furthermore, well-annotated data sets are also in short supply in the field of digital pathology. Inspired by the pathologist's clinical di…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Xiangya Hospital, Central South University（中南大学湘雅医院）；University of Chinese Academy of Sciences（中国科学院大学）；Central South University（中南大学）
Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient's paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable sampling error is risky due to tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&E pathological whole slide images (WSI) using AI method is useful and critical to assist pathologists pre-screen proper paraffin block for IHC. It's a challenging task since only WSI level…
Cross-scale Multi-instance Learning for Pathological Image Diagnosis
Vanderbilt University（范德堡大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Vanderbilt University Medical Center（范德堡大学医学中心）；Veterans Affairs Tennessee Valley Healthcare System（田纳西河谷退伍军人事务医疗系统）
CC-DCNet: Dynamic Convolutional Neural Network with Contrastive Constraints for Identifying Lung Cancer Subtypes on Multi-modality Images
Zhejiang Lab（之江实验室）；Institute of Computer Graphics and Vision（计算机图形与视觉研究所）；Graz University of Technology（格拉茨工业大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Northwestern Polytechnical University（西北工业大学）；Zhejiang University School of Medicine（浙江大学医学院）；Sir Run Run Shaw Hospital（邵逸夫医院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
A Weakly Supervised Segmentation Network Embedding Cross-scale Attention Guidance and Noise-sensitive Constraint for Detecting Tertiary Lymphoid Structures of Pancreatic Tumors
Nanjing University of Science and Technology（南京理工大学）；Nanjing University（南京大学）；Nanjing Drum Tower Hospital（南京鼓楼医院）；Affiliated Hospital of Nanjing University of Chinese Medicine（南京中医药大学附属医院）
We propose a novel solution for volumetric ultrasound imaging using single-side access 3-D synthetic-aperture scanning of a clinical linear array. This solution is based on an advanced scanning geometry and a software-based ultrasound platform. The rotate-translate scanning scheme increases the elevation angular aperture by pivoting the array [-45{\textdegree} to 45{\textdegree}] around its array axis (axis along the row of its elements) and then, scans the imaged object for each pivoted angle by translating the array perpendicularly to the rotation axis. A theoretical basis is presented so th…
The Athenian School（雅典纳中学）；Cornell University（康奈尔大学）
Because of the necessity to obtain high-quality images with minimal radiation doses, such as in low-field magnetic resonance imaging, super-resolution reconstruction in medical imaging has become more popular (MRI). However, due to the complexity and high aesthetic requirements of medical imaging, image super-resolution reconstruction remains a difficult challenge. In this paper, we offer a deep learning-based strategy for reconstructing medical images from low resolutions utilizing Transformer and Generative Adversarial Networks (T-GAN). The integrated system can extract more precise texture …
Super-resolution techniques overcome the diffraction-limit and get very high resolutions. A category of these techniques, e.g., STED achieves this by creating an illumination spot smaller than the Airy Disk. As a result, points are distinguishable even if they are as small as the spot. In order to further observe structures smaller than the spot itself, a technique called DDS scans the sample more densely, and recovers the expected image by deconvolution. In that technique, the deconvolution is achieved by filtering which requires some peripheral areas to be scanned together with the region of…
Genentech（基因泰克）；University of Central Florida（中佛罗里达大学）
The integration of multi-modal data, such as pathological images and genomic data, is essential for understanding cancer heterogeneity and complexity for personalized treatments, as well as for enhancing survival predictions. Despite the progress made in integrating pathology and genomic data, most existing methods cannot mine the complex inter-modality relations thoroughly. Additionally, identifying explainable features from these models that govern preclinical discovery and clinical prediction is crucial for cancer diagnosis, prognosis, and therapeutic response studies. We propose PONET- a n…
