Segmentation-free integration of nuclei morphology and spatial transcriptomics for retinal images
Dept. of Information Technology（信息技术系）；Dept. of Molecular, Cell and Developmental Biology（分子、细胞与发育生物学系）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Eduard Chelebian, Pratiti Dasgupta, Zainalabedin Samadi, Carolina Wählby, Amjad Askary
This study introduces SEFI (SEgmentation-Free Integration), a novel method for integrating morphological features of cell nuclei with spatial transcriptomics data. Cell segmentation poses a significant challenge in the analysis of spatial transcriptomics data, as tissue-specific structural complexities and densely packed cells in certain regions make it difficult to develop a universal approach. SEFI addresses this by utilizing self-supervised learning to extract morphological features from fluorescent nuclear staining images, enhancing the clustering of gene expression data without requiring segmentation. We demonstrate SEFI on spatially resolved gene expression profiles of the developing retina, acquired using multiplexed single molecule Fluorescence In Situ Hybridization (smFISH). SEFI is publicly available at https://github.com/eduardchelebian/sefi.
Enhancement, followed by segmentation, quantification and modelling, of blood vessels in retinal images plays an essential role in computer-aid retinopathy diagnosis. In this paper, we introduce a new vessel enhancement method which is the bowler-hat transform based on mathematical morphology. The proposed method combines different structuring elements to detect innate features of vessel-like structures. We evaluate the proposed method qualitatively and quantitatively, and compare it with the existing, state-of-the-art methods using both synthetic and real datasets. Our results show that the p…
Medical University of Vienna（维也纳医科大学）；Centre for Eye Research Australia（澳大利亚眼科研究中心）；Royal Victorian Eye and Ear Hospital（皇家维多利亚眼耳医院）；The University of Melbourne（墨尔本大学）
Longitudinal imaging is capable of capturing the static ana\-to\-mi\-cal structures and the dynamic changes of the morphology resulting from aging or disease progression. Self-supervised learning allows to learn new representation from available large unlabelled data without any expert knowledge. We propose a deep learning self-supervised approach to model disease progression from longitudinal retinal optical coherence tomography (OCT). Our self-supervised model takes benefit from a generic time-related task, by learning to estimate the time interval between pairs of scans acquired from the sa…
PAST: A multimodal single-cell foundation model for histopathology and spatial transcriptomics in cancer
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；Shanghai Academy of Artificial Intelligence for Science（上海科学智能研究院）；Fudan University（复旦大学）；Zhongshan Hospital, Fudan University（复旦大学附属中山医院）
The morphology of retinal blood vessels can indicate various diseases in the human body, and researchers have been working on automatic scanning and segmentation of retinal images to aid diagnosis. This project compares the performance of four neural network architectures in segmenting retinal images, using a combined dataset from different databases, namely the UNet, DR-VNet, UNet-ResNet and UNet-VGG.
Karolinska Institutet（卡罗林斯卡学院）；Tampere University（坦佩雷大学）；Lund University（隆德大学）；University of Turku（图尔库大学）；Karolinska University Laboratory（卡罗林斯卡大学实验室）
Molecular phenotyping is central in cancer precision medicine, but remains costly and standard methods only provide a tumour average profile. Microscopic morphological patterns observable in histopathology sections from tumours are determined by the underlying molecular phenotype and associated with clinical factors. The relationship between morphology and molecular phenotype has a potential to be exploited for prediction of the molecular phenotype from the morphology visible in histopathology images. We report the first transcriptome-wide Expression-MOrphology (EMO) analysis in breast cancer,…
A Deep Learning-Based Unified Framework for Red Lesions Detection on Retinal Fundus Images
Computer and Information Science College, King Saud University（沙特王后大学计算机与信息科学学院）；Department of Ophthalmology, College of Medicine, Princess Nourah bint Abdulrahman University（普里丝特努尔·阿卜杜勒拉赫曼大学医学院眼科系）
Computer vision and image processing techniques provide important assistance to physicians and relieve their workload in different tasks. In particular, identifying objects of interest such as lesions and anatomical structures from the image is a challenging and iterative process that can be done by using computer vision and image processing approaches in a successful manner. Optic Nerve Head (ONH) detection is a crucial step in retinal image analysis algorithms. The goal of ONH detection is to find and detect other retinal landmarks and lesions and their corresponding diameters, to use as a l…
Many eye diseases like Diabetic Macular Edema (DME), Age-related Macular Degeneration (AMD), and Glaucoma manifest in the retina, can cause irreversible blindness or severely impair the central version. The Optical Coherence Tomography (OCT), a 3D scan of the retina with high qualitative information about the retinal morphology, can be used to diagnose and monitor changes in the retinal anatomy. Many Deep Learning (DL) methods have shared the success of developing an automated tool to monitor pathological changes in the retina. However, the success of these methods depend mainly on large datas…
Bridging the gap between Performance and Interpretability: An Explainable Disentangled Multimodal Framework for Cancer Survival Prediction
AI Technology for Life（人工智能技术与生命科学）；Department of Information and Computing Sciences（信息与计算科学系）；Department of Biology（生物学系）；Utrecht University（乌得勒支大学）；Department of Metabolic Diseases（代谢疾病部门）；Wilhelmina Children’s Hospital（维廉明娜儿童医院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Regenerative Medicine Center Utrecht（乌得勒支再生医学中心）；Computational Pathology（计算病理学）；Department of Pathology（病理学系）；The Netherlands Cancer Institute（荷兰癌症研究所）；Visualization and Graphics（可视化与图形学）；The Netherlands Cancer Institute, Amsterdam（荷兰癌症研究所，阿姆斯特丹）
Adaptive Multi-Scale Integration Unlocks Robust Cell Annotation in Histopathology Images
Department of Computer and Information Science, University of Pennsylvania（计算机与信息科学系，宾夕法尼亚大学）；Department of Bioengineering, University of Pennsylvania（生物工程系，宾夕法尼亚大学）；Department of Pathology and Laboratory Medicine, University of Pennsylvania（病理学与实验室医学系，宾夕法尼亚大学）；Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania（生物统计学、流行病学与信息学系，宾夕法尼亚大学）
National Heart and Lung Institute, Faculty of Medicine, Imperial College London（帝国理工学院医学院国家心肺研究所）；Department of Bioengineering, Faculty of Engineering, Imperial College London（帝国理工学院工程系生物工程系）；Royal Brompton Hospital（皇家布朗普顿医院）
Various structures in human physiology follow a treelike morphology, which often expresses complexity at very fine scales. Examples of such structures are intrathoracic airways, retinal blood vessels, and hepatic blood vessels. Large collections of 2D and 3D images have been made available by medical imaging modalities such as magnetic resonance imaging (MRI), computed tomography (CT), Optical coherence tomography (OCT) and ultrasound in which the spatial arrangement can be observed. Segmentation of these structures in medical imaging is of great importance since the analysis of the structure …
Fine-tuning Vision Language Models with Graph-based Knowledge for Explainable Medical Image Analysis
School of Electrical and Computer Engineering, Cornell University（电气与计算机工程学院，康奈尔大学）；School of Computation, Information and Technology, Technical University of Munich（计算、信息与技术学院，慕尼黑技术大学）；Cornell Tech（康奈尔科技）；Weill Cornell Medicine（韦尔·康奈尔医学）
VAMPIRE: Uncovering Vessel Directional and Morphological Information from OCTA Images for Cardiovascular Disease Risk Factor Prediction
The Hong Kong University of Science and Technology, Hong Kong, China（香港科学与技术大学）；Department of Radiology, Guangdong Provincial People's Hospital (Guangdong Academy of Medical Sciences), Southern Medical University, Guangzhou, China（广东省人民医院放射科（广东省医学科学院）南方医科大学）；Health Management Center, Foshan First People's Hospital（佛山第一人民医院健康管理中心）；Health Management Center, The Sixth Affiliated Hospital, School of Medicine, South China University of Technology（健康管理中心，华南理工大学医学院第六附属医院）
Simon Fraser University（西蒙弗雷泽大学）；University of British Columbia（不列颠哥伦比亚大学）；Medical University of Vienna（维也纳医科大学）；University of Zagreb（萨格勒布大学）
Purpose: Optical Coherence Tomography Angiography (OCT-A) permits visualization of the changes to the retinal circulation due to diabetic retinopathy (DR), a microvascular complication of diabetes. We demonstrate accurate segmentation of the vascular morphology for the superficial capillary plexus and deep vascular complex (SCP and DVC) using a convolutional neural network (CNN) for quantitative analysis. Methods: Retinal OCT-A with a 6x6mm field of view (FOV) were acquired using a Zeiss PlexElite. Multiple-volume acquisition and averaging enhanced the vessel network contrast used for training…
Southern University of Science and Technology（南方科技大学）；Sun Yat-Sen University（中山大学）；Hunan University of Arts and Science（湖南文理学院）
Optical coherence tomography angiography (OCTA) is a novel non-invasive imaging technique that allows visualizations of vasculature and foveal avascular zone (FAZ) across retinal layers. Clinical researches suggest that the morphology and contour irregularity of FAZ are important biomarkers of various ocular pathologies. Therefore, precise segmentation of FAZ has great clinical interest. Also, there is no existing research reporting that FAZ features can improve the performance of deep diagnostic classification networks. In this paper, we propose a novel multi-level boundary shape and distance…
