arXiv 2406.172252024-06-25eess.IVcs.CV

用于全切片病理图像生存分析的多模态跨任务交互

Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images

School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；School of Science, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）理学院）

Songhan Jiang, Zhengyu Gan, Linghan Cai, Yifeng Wang, Yongbing Zhang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出多模态跨任务交互（MCTI）框架，利用亚型分类任务挖掘肿瘤区域并引入传输引导注意力模块建模任务间关联，有效融合病理图像与基因组数据，在三个公开基准上提升了生存预测性能。

摘要：

利用病理图像和基因组谱进行生存预测在癌症分析和预后中日益重要。尽管取得了显著进展，精确的生存分析仍面临两大挑战：（1）全切片图像（WSIs）包含海量像素，使病理图像处理复杂化，难以生成有效的肿瘤微环境（TME）表示。（2）现有多模态方法常依赖对齐策略整合互补信息，由于病理与基因间固有的异质性，可能导致信息丢失。本文提出多模态跨任务交互（MCTI）框架，探索亚型分类与生存分析任务间的内在关联。具体而言，为捕获WSIs中TME相关特征，我们利用亚型分类任务挖掘肿瘤区域。同时，在基因组特征提取中应用多头注意力机制，自适应执行基因分组以获取任务相关的基因组嵌入。结合病理图像与基因组数据的联合表示，我们进一步引入传输引导注意力（TGA）模块，利用最优传输理论建模亚型分类与生存分析任务间的关联，有效传递潜在信息。大量实验证明了本方法的优越性，MCTI在三个公开基准上优于现有最先进框架。

英文摘要：

Survival prediction, utilizing pathological images and genomic profiles, is increasingly important in cancer analysis and prognosis. Despite significant progress, precise survival analysis still faces two main challenges: (1) The massive pixels contained in whole slide images (WSIs) complicate the process of pathological images, making it difficult to generate an effective representation of the tumor microenvironment (TME). (2) Existing multimodal methods often rely on alignment strategies to integrate complementary information, which may lead to information loss due to the inherent heterogeneity between pathology and genes. In this paper, we propose a Multimodal Cross-Task Interaction (MCTI) framework to explore the intrinsic correlations between subtype classification and survival analysis tasks. Specifically, to capture TME-related features in WSIs, we leverage the subtype classification task to mine tumor regions. Simultaneously, multi-head attention mechanisms are applied in genomic feature extraction, adaptively performing genes grouping to obtain task-related genomic embedding. With the joint representation of pathological images and genomic data, we further introduce a Transport-Guided Attention (TGA) module that uses optimal transport theory to model the correlation between subtype classification and survival analysis tasks, effectively transferring potential information. Extensive experiments demonstrate the superiority of our approaches, with MCTI outperforming state-of-the-art frameworks on three public benchmarks. \href{https://github.com/jsh0792/MCTI}{https://github.com/jsh0792/MCTI}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。