一种用于泛癌预后的多模态深度学习框架
A Multi-Modal Deep Learning Framework for Pan-Cancer Prognosis
AI总结：
针对现有泛癌预后模型模态单一、泛化性弱的问题，提出多模态深度学习框架UMPSNet，融合病理、基因、文本等多模态特征，结合OT注意力与GMoE机制实现单模型多癌预后，性能优于当前最优方法。
摘要：
预后任务与患者生存分析、治疗方案优化及资源分配密切相关，具有重要意义。现有预后模型在特定数据集上已取得不错的结果，但存在两方面局限：一方面，它们仅探究了特定类型的模态数据，例如患者组织病理学WSI（全切片图像）和基因表达分析；另一方面，它们采用“每种癌症对应一个模型”的范式，即训练出的模型仅能预测单一癌症类型的预后效果，导致泛化能力较弱。\n本文提出了一种基于深度学习的模型，名为UMPSNet。具体而言，为全面了解患者病情，除分别构建组织病理学图像编码器和基因组表达谱编码器外，UMPSNet还将四类重要元数据（人口统计学信息、癌症类型信息、治疗方案和诊断结果）整合到文本模板中，随后引入文本编码器提取文本特征。此外，该模型利用基于最优传输OT的注意力机制对不同模态的特征进行对齐和融合。进一步地，引入引导式软混合专家（GMoE）机制，有效解决多癌症数据集间的分布差异问题。\n通过整合患者数据的多模态信息并进行联合训练，UMPSNet的性能优于所有SOTA（当前最优）方法，并且证明了所提出的“单模型适用于多种癌症类型”学习范式的有效性与泛化能力。UMPSNet的代码可在https://github.com/binging512/UMPSNet获取。
英文摘要：
Prognostic task is of great importance as it closely related to the survival analysis of patients, the optimization of treatment plans and the allocation of resources. The existing prognostic models have shown promising results on specific datasets, but there are limitations in two aspects. On the one hand, they merely explore certain types of modal data, such as patient histopathology WSI and gene expression analysis. On the other hand, they adopt the per-cancer-per-model paradigm, which means the trained models can only predict the prognostic effect of a single type of cancer, resulting in weak generalization ability. In this paper, a deep-learning based model, named UMPSNet, is proposed. Specifically, to comprehensively understand the condition of patients, in addition to constructing encoders for histopathology images and genomic expression profiles respectively, UMPSNet further integrates four types of important meta data (demographic information, cancer type information, treatment protocols, and diagnosis results) into text templates, and then introduces a text encoder to extract textual features. In addition, the optimal transport OT-based attention mechanism is utilized to align and fuse features of different modalities. Furthermore, a guided soft mixture of experts (GMoE) mechanism is introduced to effectively address the issue of distribution differences among multiple cancer datasets. By incorporating the multi-modality of patient data and joint training, UMPSNet outperforms all SOTA approaches, and moreover, it demonstrates the effectiveness and generalization ability of the proposed learning paradigm of a single model for multiple cancer types. The code of UMPSNet is available at https://github.com/binging512/UMPSNet.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
