从基因名称、表达值和图像中学习：用于空间转录组学表示学习的对比掩码文本-图像预训练
Learning from Gene Names, Expression Values and Images: Contrastive Masked Text-Image Pretraining for Spatial Transcriptomics Representation Learning
AI总结：
本文提出CoMTIP框架，通过对比掩码文本-图像预训练联合学习图像、基因名称和表达值，并引入掩码特征建模与成对感知对抗训练，在空间转录组学下游任务中超越现有方法并实现零样本基因表达预测。
摘要：
空间转录组学旨在将高分辨率组织学图像与空间解析的基因表达连接起来。为了在下游任务（如基因表达预测）中取得更好的性能，需要进行大规模预训练，以获得能够跨组织、实验方案和实验室桥接组织学与转录组学的可泛化表示。现有的空间转录组学跨模态预训练方法要么孤立地依赖基因名称，要么孤立地依赖表达值，这剥夺了基因分支的基本语义，并破坏了每个基因与其定量数值之间的关联。此外，由于将监督限制在图像-文本对齐上，这些方法忽略了对于学习鲁棒图像特征至关重要的内在视觉线索。我们提出了CoMTIP，这是第一个对比掩码文本-图像预训练框架，它联合从图像、基因名称和表达值中学习，同时为空间转录组学捕获细粒度视觉上下文。视觉分支使用掩码特征建模来重建被遮挡的图像块，并学习上下文感知的图像嵌入。文本分支采用可扩展的基因-文本编码器，并行处理所有基因句子，通过专用嵌入丰富每个基因及其数值，并采用成对感知对抗训练（PAAT）来保持正确的基因-值关联。图像和文本表示在共享的InfoNCE优化空间中对齐。在公开空间转录组学数据集上的实验表明，CoMTIP不仅在多种下游任务上超越了先前的方法，而且实现了零样本基因表达预测，这是现有方法所不具备的能力。
英文摘要：
Spatial transcriptomics aims to connect high-resolution histology images with spatially resolved gene expression. To achieve better performance on downstream tasks such as gene expression prediction, large-scale pre-training is required to obtain generalisable representations that can bridge histology and transcriptomics across tissues, protocols, and laboratories. Existing cross-modal pre-training approaches for spatial transcriptomics rely on either gene names or expression values in isolation, which strips the gene branch of essential semantics and breaks the association between each gene and its quantitative magnitude. In addition, by restricting supervision to image-text alignment, these methods ignore intrinsic visual cues that are critical for learning robust image features. We present CoMTIP, the first Contrastive Masked Text-Image Pretraining framework that jointly learns from images, gene names, and expression values while capturing fine-grained visual context for spatial transcriptomics. The vision branch uses Masked Feature Modeling to reconstruct occluded patches and learn context-aware image embeddings. The text branch applies a scalable Gene-Text Encoder that processes all gene sentences in parallel, enriches each gene and its numerical value with dedicated embeddings, and employs Pair-aware Adversarial Training (PAAT) to preserve correct gene-value associations. Image and text representations are aligned in a shared InfoNCE-optimised space. Experiments on public spatial transcriptomics datasets show that CoMTIP not only surpasses previous methods on diverse downstream tasks but also achieves zero-shot gene expression prediction, a capability that existing approaches do not provide.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
