计算病理学中转录组学引导的切片表征学习
Transcriptomics-guided Slide Representation Learning in Computational Pathology
AI总结：
提出 Tangle 多模态预训练方法，利用基因表达谱通过对比学习对齐全切片图像与转录组表征，在多器官少样本分类和检索中显著优于基线。
摘要：
自监督学习（SSL）已成功用于构建小型组织学图像（例如224×224像素）的图像块嵌入，但将这些模型扩展到从整张千兆像素全切片图像（WSI）中学习切片嵌入仍具挑战性。本文利用基因表达谱中的互补信息，通过多模态预训练来引导切片表征学习。表达谱构成了组织的高度详细分子描述，我们假设其可为学习切片嵌入提供强大的任务无关训练信号。我们的切片与表达（S+E）预训练策略名为 Tangle，采用特定模态编码器，并通过对比学习对齐其输出。Tangle 在来自两个不同物种（智人和褐家鼠）的三个不同器官样本上进行预训练：肝脏（n=6,597 个 S+E 对）、乳腺（n=1,020）和肺（n=1,012）。在由1,265张乳腺 WSI、1,946张肺 WSI 和4,584张肝脏 WSI 组成的三个独立测试数据集上，Tangle 的少样本性能显著优于监督基线和 SSL 基线。在使用基于原型的分类和切片检索进行评估时，Tangle 相较所有基线也表现出显著性能提升。代码位于 https://github.com/mahmoodlab/TANGLE。
英文摘要：
Self-supervised learning (SSL) has been successful in building patch embeddings of small histology images (e.g., 224x224 pixels), but scaling these models to learn slide embeddings from the entirety of giga-pixel whole-slide images (WSIs) remains challenging. Here, we leverage complementary information from gene expression profiles to guide slide representation learning using multimodal pre-training. Expression profiles constitute highly detailed molecular descriptions of a tissue that we hypothesize offer a strong task-agnostic training signal for learning slide embeddings. Our slide and expression (S+E) pre-training strategy, called Tangle, employs modality-specific encoders, the outputs of which are aligned via contrastive learning. Tangle was pre-trained on samples from three different organs: liver (n=6,597 S+E pairs), breast (n=1,020), and lung (n=1,012) from two different species (Homo sapiens and Rattus norvegicus). Across three independent test datasets consisting of 1,265 breast WSIs, 1,946 lung WSIs, and 4,584 liver WSIs, Tangle shows significantly better few-shot performance compared to supervised and SSL baselines. When assessed using prototype-based classification and slide retrieval, Tangle also shows a substantial performance improvement over all baselines. Code available at https://github.com/mahmoodlab/TANGLE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
