BiTro:双向迁移学习增强癌症病理图像的批量和空间转录组预测
BiTro: Bidirectional Transfer Learning Enhances Bulk and Spatial Transcriptomics Prediction in Cancer Pathological Images
AI总结:
本文提出BiTro框架,通过双向迁移学习提升病理图像中批量和空间转录组预测的准确性,设计了适用于两种数据的通用模型,并利用LoRA实现跨数据集的高效迁移。
摘要:
癌症病理分析需要建模多种模态下的肿瘤异质性,主要通过转录组和全片成像(WSI)及其空间关系。一方面,批量转录组和WSI图像虽丰富但缺乏空间映射;另一方面,空间转录组(ST)数据提供高空间分辨率,但面临成本高、测序深度低和样本量少的挑战。因此,单侧数据基础存在缺陷,难以准确建立两种模态的映射。为此,我们提出BiTro,一种双向迁移学习框架,可增强病理图像中批量和空间转录组的预测。我们的贡献是双方面的:首先,我们设计了一个通用且可迁移的模型架构,适用于批量+WSI和ST数据。主要亮点是将WSI图像建模在细胞层面以更好地捕捉细胞的视觉特征、形态表型及其空间关系;为了将细胞特征映射到其在批量或ST中的转录组测得值,我们采用多实例学习。其次,通过使用LoRA,我们的模型可以在批量和ST数据之间高效迁移,以利用其互补信息。为了测试我们的框架,我们在五个癌症数据集中进行了全面实验。结果表明,1)我们的基模型在批量或空间转录组预测中可实现更好或具有竞争力的性能;2)迁移学习可进一步提升基模型的性能。
英文摘要:
Cancer pathological analysis requires modeling tumor heterogeneity across multiple modalities, primarily through transcriptomics and whole slide imaging (WSI), along with their spatial relations. On one hand, bulk transcriptomics and WSI images are largely available but lack spatial mapping; on the other hand, spatial transcriptomics (ST) data can offer high spatial resolution, yet facing challenges of high cost, low sequencing depth, and limited sample sizes. Therefore, the data foundation of either side is flawed and has its limit in accurately finding the mapping between the two modalities. To this end, we propose BiTro, a bidirectional transfer learning framework that can enhance bulk and spatial transcriptomics prediction from pathological images. Our contributions are twofold. First, we design a universal and transferable model architecture that works for both bulk+WSI and ST data. A major highlight is that we model WSI images on the cellular level to better capture cells' visual features, morphological phenotypes, and their spatial relations; to map cells' features to their transcriptomics measured in bulk or ST, we adopt multiple instance learning. Second, by using LoRA, our model can be efficiently transferred between bulk and ST data to exploit their complementary information. To test our framework, we conducted comprehensive experiments on five cancer datasets. Results demonstrate that 1) our base model can achieve better or competitive performance compared to existing models on bulk or spatial transcriptomics prediction, and 2) transfer learning can further improve the base model's performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。