Integrating histopathology with spatial transcriptomics (ST) provides a powerful opportunity to link tissue morphology with molecular function. Yet most existing multimodal approaches rely on a small set of highly variable genes, which limits predictive scope and overlooks the coordinated biological programs that shape tissue phenotypes. We present PEaRL (Pathway Enhanced Representation Learning), a multimodal framework that represents transcriptomics through pathway activation scores computed with ssGSEA. By encoding biologically coherent pathway signals with a transformer and aligning them with histology features via contrastive learning, PEaRL reduces dimensionality, improves interpretability, and strengthens cross-modal correspondence. Across three cancer ST datasets (breast, skin, and lymph node), PEaRL consistently outperforms SOTA methods, yielding higher accuracy for both gene- and pathway-level expression prediction (up to 58.9 percent and 20.4 percent increase in Pearson correlation coefficient compared to SOTA). These results demonstrate that grounding transcriptomic representation in pathways produces more biologically faithful and interpretable multimodal models, advancing computational pathology beyond gene-level embeddings.
Integrating Pathology Foundation Models and Spatial Transcriptomics for Cellular Decomposition from Histology Images
School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA（电气与计算机工程学院，佐治亚理工学院）；Department of Biomedical Engineering, Georgia Institute of Technology（生物医学工程系，佐治亚理工学院）；Emory University, Atlanta, GA, USA（埃默里大学）
HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction
Cancer Hospital of Dalian University of Technology, Shenyang, China（大连理工大学沈阳医院）；School of Biomedical Engineering, Faculty of Medicine, Dalian University of Technology, Dalian, China（大连理工大学生物医学工程学院）；Key Laboratory of Integrated Circuit and Biomedical Electronic System, Dalian University of Technology, Dalian, China（大连理工大学集成电路与生物医学电子系统重点实验室）
Shanghai Jiao Tong University（上海交通大学）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；University of New South Wales（新南威尔士大学）；BirenTech Research（壁仞科技研究院）
Stain variations often decrease the generalization ability of deep learning based approaches in digital histopathology analysis. Two separate proposals, namely stain normalization (SN) and stain augmentation (SA), have been spotlighted to reduce the generalization error, where the former alleviates the stain shift across different medical centers using template image and the latter enriches the accessible stain styles by the simulation of more stain variations. However, their applications are bounded by the selection of template images and the construction of unrealistic styles. To address the…
Uppsala University（乌普萨拉大学）；SciLifeLab（瑞典生命科学实验室）；Radboud University Medical Center（拉德堡德大学医学中心）
Annotations are necessary to develop computer vision algorithms for histopathology, but dense annotations at a high resolution are often time-consuming to make. Deep learning models for segmentation are a way to alleviate the process, but require large amounts of training data, training times and computing power. To address these issues, we present seeded iterative clustering to produce a coarse segmentation densely and at the whole slide level. The algorithm uses precomputed representations as the clustering space and a limited amount of sparse interactive annotations as seeds to iteratively …
VitaminP: cross-modal learning enables whole-cell segmentation from routine histology
Department of Translational Molecular Pathology, Division of Pathology and Laboratory Medicine, The University of Texas MD Anderson Cancer Center（转化分子病理学部门，病理与实验室医学分会，德克萨斯大学MD安德森癌症中心）；Institute for Data Science in Oncology, The University of Texas MD Anderson Cancer Center（肿瘤数据科学研究所，德克萨斯大学MD安德森癌症中心）；The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）
Virtual staining for 3D X-ray histology of bone implants
Institute of Materials Physics（材料物理研究所）；Helmholtz-Zentrum Hereon（海德堡研究中心）；Institute of Metallic Biomaterials（金属生物材料研究所）；Bruker Daltonics SPR（布鲁克尔达特利克斯SPR）；Data-Driven Analysis and Design of Materials（数据驱动的材料分析与设计）；University of Rostock（罗斯托克大学）
Microsatellite instability (MSI) is a tumor phenotype whose diagnosis largely impacts patient care in colorectal cancers (CRC), and is associated with response to immunotherapy in all solid tumors. Deep learning models detecting MSI tumors directly from H&E stained slides have shown promise in improving diagnosis of MSI patients. Prior deep learning models for MSI detection have relied on neural networks pretrained on ImageNet dataset, which does not contain any medical image. In this study, we leverage recent advances in self-supervised learning by training neural networks on histology images…
A Generative Foundation Model for Multimodal Histopathology
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤学系）；Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine（斯坦福大学医学院精神病学与行为科学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Radiology, Stanford University（斯坦福大学放射学系）；Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）
Vanderbilt University（范德堡大学）；Middle Tennessee State University（中田纳西州立大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for non-invasive imaging of the local fiber architecture of the human brain at a millimetric scale. Multiple classical approaches have been proposed to detect both single (e.g., tensors) and multiple (e.g., constrained spherical deconvolution, CSD) fiber population orientations per voxel. However, existing techniques generally exhibit low reproducibility across MRI scanners. Herein, we propose a data-driven tech-nique using a neural network design which exploits two categories of data. First, training data were acquired on three squ…
HistoColAi: An Open-Source Web Platform for Collaborative Digital Histology Image Annotation with AI-Driven Predictive Integration
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València（瓦伦西亚理工大学以人为中心技术大学研究院）；valgrAI: Valencian Graduate School and Research Network of Artificial Intelligence（瓦伦西亚人工智能研究生院及研究网络）；ÉTS Montréal（蒙特利尔工程学院）
University of Warwick（华威大学）；Tissue Image Analytics (TIA) Centre, University of Warwick（华威大学组织图像分析中心）
The problem of recognizing various types of tissues present in multi-gigapixel histology images is an important fundamental pre-requisite for downstream analysis of the tumor microenvironment in a bottom-up analysis paradigm for computational pathology. In this paper, we propose a deep dictionary learning approach to solve the problem of tissue phenotyping in histology images. We propose deep Multi-Resolution Dictionary Learning (deepMRDL) in order to benefit from deep texture descriptors at multiple different spatial resolutions. We show the efficacy of the proposed approach through extensive…
University of Warwick（华威大学）；GlaxoSmithKline（葛兰素史克）
"Is it possible to predict expression levels of different genes at a given spatial location in the routine histology image of a tumor section by modeling its stain absorption characteristics?" In this work, we propose a "stain-aware" machine learning approach for prediction of spatial transcriptomic gene expression profiles using digital pathology image of a routine Hematoxylin & Eosin (H&E) histology section. Unlike recent deep learning methods which are used for gene expression prediction, our proposed approach termed Neural Stain Learning (NSL) explicitly models the association of stain abs…
Img2ST-Net: Efficient High-Resolution Spatial Omics Prediction from Whole Slide Histology Images via Fully Convolutional Image-to-Image Learning
Department of Computer Science, Vanderbilt University（计算机科学系，范德比尔特大学）；Weill Cornell Medicine（韦尔·科恩医学中心）；Department of Electrical and Computer Engineering, Vanderbilt University（电气与计算机工程系，范德比尔特大学）；Department of Biostatistics, Vanderbilt University Medical Center（生物统计学系，范德比尔特大学医学中心）；NVIDIA
Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；University Health Network（大学健康网络）；The Donnelly Centre（唐纳利中心）；Toronto General Hospital Research Institute（多伦多总医院研究所）；Canadian Institute for Advanced Research (CIFAR)（加拿大高等研究院（CIFAR））
3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology
Cambridge Stem Cell Institute, University of Cambridge（剑桥干细胞研究所，剑桥大学）；Cambridge Center for AI in Medicine, University of Cambridge（剑桥人工智能医学中心，剑桥大学）；Department of Medicine, University of Cambridge（剑桥大学医学系）；Computer Science Department, University of Oxford（牛津大学计算机科学系）；Computer Science Department, University of Helsinki（赫尔辛基大学计算机科学系）；Institute for Molecular Medicine Finland, University of Helsinki（芬兰分子医学研究所，赫尔辛基大学）；Wellcome Sanger Institute（沃森桑格研究所）
GC-MoE: Genomics-Guided Cell-Type-Specific Mixture of Experts for Histology-Based Single-Cell Spatial Transcriptomics
Kyushu University（九州大学）；German Research Center for Artificial Intelligence (DFKI GmbH)（德国人工智能研究中心）；RPTU University Kaiserslautern-Landau（科布伦茨-劳恩堡大学）；The University of Osaka（大阪大学）；IntelligentX GmbH；Osaka Metropolitan University（大阪 Metropolitan 大学）
Image Data Analytics, Informatics IT, Merck \& Co., Inc., West Point, PA 19486, USA Image Data Analytics, Informatics IT, Merck \& Co., Inc., Rahway, NJ 07065, USA
It is common for pathologists to annotate specific regions of the tissue, such as tumor, directly on the glass slide with markers. Although this practice was helpful prior to the advent of histology whole slide digitization, it often occludes important details which are increasingly relevant to immuno-oncology due to recent advancements in digital pathology imaging techniques. The current work uses a generative adversarial network with cycle loss to remove these annotations while still maintaining the underlying structure of the tissue by solving an image-to-image translation problem. We train…
University of Houston（休斯顿大学）；University Hospital of Cologne（科隆大学医院）
The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within) pathology labs, coupled with the gradual fading of archival sections. Variations in stain colors and intensities can make tissue evaluation difficult for pathologists, ultimately affecting their abi…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California NanoSystems Institute（加州纳米系统研究所）；David Geffen School of Medicine（戴维·格芬医学院）
Histological analysis of tissue samples is one of the most widely used methods for disease diagnosis. After taking a sample from a patient, it goes through a lengthy and laborious preparation, which stains the tissue to visualize different histological features under a microscope. Here, we demonstrate a label-free approach to create a virtually-stained microscopic image using a single wide-field auto-fluorescence image of an unlabeled tissue sample, bypassing the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses a convolutional neura…
CHRep: Cross-modal Histology Representation and Post-hoc Calibration for Spatial Gene Expression Prediction
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Key Laboratory of Network System and Network Culture（北京网络系统与网络文化重点实验室）；Peking University Third Hospital（北京大学第三医院）
Cross-modal Diffusion Modelling for Super-resolved Spatial Transcriptomics
Department of Clinical Neurosciences, University of Cambridge, UK（剑桥大学临床神经科学系）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（剑桥大学应用数学与理论物理系）；School of Science and Engineering, University of Dundee, UK（邓迪大学科学与工程学院）；School of Medicine, University of Dundee, UK（邓迪大学医学院）；Zhejiang Lab, China（浙江实验室）
Radboud University Medical Center（拉德堡德大学医学中心）；Canisius-Wilhelmina Hospital（卡尼修斯-威廉明娜医院）；Jeroen Bosch Hospital（耶伦博斯医院）
Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by: (1) noisy and expensive reference standards established by pathologists, (2) lack of generalization due to staining variation across laboratories, and (3) high compu…
High-Resolution Spatial Transcriptomics from Histology Images using HisToSGE
School of Information Science and Engineering, Yunnan University（云南大学信息科学与工程学院）；College of Nursing Health Sciences, Yunnan Open University（云南开放大学护理与健康科学学院）
Score-based Diffusion Model for Unpaired Virtual Histology Staining
Department of Health Technology and Informatics, Hong Kong Polytechnic University, China（香港理工大学健康科技与信息学系）；Department of Clinical Neurosciences, University of Cambridge, UK（剑桥大学临床神经科学系）；School of Science and Engineering, University of Dundee, UK（邓迪大学科学与工程学院）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge, UK（剑桥大学应用数学与理论物理系）
