AI 中文总结
本研究针对临床信息未充分利用及多模态异质性问题，提出CIGTSurv框架，结合局部原型关联与全局特征对齐机制，在五个TCGA癌症队列上取得生存预测SOTA性能。
AI 中文摘要
多模态学习通过整合病理图像与基因组数据，大幅推动了生存预测的发展。然而，临床信息虽对反映患者整体健康状况至关重要，却因具有离散、稀疏、低维的特性而未得到充分利用。此外，各模态间固有的异质性给跨模态交互建模带来了重大挑战。本文提出CIGTSurv，这是一种临床信息引导的三模态生存预测框架。具体而言，我们首先设计了一个整体文本模板，并使用预训练的基础模型将临床表格数据转换为高维标记嵌入。以临床信息为锚点，我们引入了两级交互机制：1）基于交叉注意力的局部原型关联（LPA）模块，用于显式学习不同模态间的标记级对应关系；2）基于最大均值差异（MMD）的全局特征对齐（GFA）损失，用于隐式增强跨模态分布一致性。在五个TCGA癌症队列上开展的大量实验表明，CIGTSurv实现了最先进（SOTA）的生存预测性能。我们的源代码可在该httpsURL公开获取。
英文摘要
Multimodal learning has significantly advanced survival prediction by integrating pathology images with genomic data. However, clinical information, despite its critical role in reflecting a patient' s overall health, remains underutilized due to its discrete, sparse, and low-dimensional nature. Furthermore, the inherent heterogeneity across these modalities pose significant challenges in modeling cross-modal interactions. In this paper, we propose CIGTSurv, a Clinical Information Guided Tri-modal framework for Survival prediction. Specifically, we first design a holistic text template and use pretrained foundation models to transform clinical tabular data into high-dimensional tokenized embeddings. Using clinical information as an anchor, we then introduce a dual-level interaction mechanism: 1) a local prototype association (LPA) module based on cross-attention to explicitly learn token-level correspondences between different modalities, and 2) a global feature alignment (GFA) loss based on Maximum Mean Discrepancy (MMD) to implicitly enhance cross-modal distribution consistency. Extensive experiments on five TCGA cancer cohorts demonstrate that CIGTSurv achieves state-of-the-art (SOTA) survival prediction performance. Our source code is publicly available at https://github.com/Daijing-ai/CIGT-Surv.git.
CommentsAccepted at MICCAI 2026
