arXiv 2506.193242025-06-24cs.CV

记忆增强的不完整多模态生存预测：基于跨切片与基因注意力超图学习

Memory-Augmented Incomplete Multimodal Survival Prediction via Cross-Slide and Gene-Attentive Hypergraph Learning

Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；School of Software, Tsinghua University（清华大学软件学院）；School of Computer Science and Engineering, UNSW Sydney（新南威尔士大学计算机科学与工程学院）

Mingcheng Qu, Guang Yang, Donglin Di, Yue Gao, Tonghua Su, Yang Song, Lei Fan

备注：: accepted by MICCAI2025 code: https://github.com/MCPathology/M2Surv

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对多模态病理-基因组生存预测中模态不平衡与数据不完整问题，提出基于超图学习和记忆机制的框架，在TCGA数据集上C-Index提升超2.3%，不完整模态下优于单模态模型。

摘要：

多模态病理-基因组分析对于癌症生存预测至关重要。然而，现有方法主要整合福尔马林固定石蜡包埋（FFPE）切片与基因组数据，而忽略了其他保存类型切片（如新鲜冷冻（FF）切片）的可用性。此外，由于病理数据的高分辨率空间特性往往主导跨模态融合过程，这阻碍了有效的多模态融合，并导致病理与基因组之间的模态不平衡挑战。这些方法通常还需要完整的数据模态，限制了其在模态不完整（如缺失病理或基因组数据）情况下的临床适用性。在本文中，我们提出了一种多模态生存预测框架，利用超图学习有效整合多WSI信息以及病理切片与基因组数据之间的跨模态交互，同时解决模态不平衡问题。此外，我们引入了一种记忆机制，存储先前学习到的配对病理-基因组特征，并动态补偿不完整的模态。在五个TCGA数据集上的实验表明，我们的模型在C-Index上比先进方法高出超过2.3%。在不完整模态场景下，我们的方法超越了仅病理模型（3.3%）和仅基因模型（7.9%）。代码：https://github.com/MCPathology/M2Surv

英文摘要：

Multimodal pathology-genomic analysis is critical for cancer survival prediction. However, existing approaches predominantly integrate formalin-fixed paraffin-embedded (FFPE) slides with genomic data, while neglecting the availability of other preservation slides, such as Fresh Froze (FF) slides. Moreover, as the high-resolution spatial nature of pathology data tends to dominate the cross-modality fusion process, it hinders effective multimodal fusion and leads to modality imbalance challenges between pathology and genomics. These methods also typically require complete data modalities, limiting their clinical applicability with incomplete modalities, such as missing either pathology or genomic data. In this paper, we propose a multimodal survival prediction framework that leverages hypergraph learning to effectively integrate multi-WSI information and cross-modality interactions between pathology slides and genomics data while addressing modality imbalance. In addition, we introduce a memory mechanism that stores previously learned paired pathology-genomic features and dynamically compensates for incomplete modalities. Experiments on five TCGA datasets demonstrate that our model outperforms advanced methods by over 2.3% in C-Index. Under incomplete modality scenarios, our approach surpasses pathology-only (3.3%) and gene-only models (7.9%). Code: https://github.com/MCPathology/M2Surv

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。