发表机构
Université Lumière Lyon 2; Léon Bérard Center; EPITA（里昂第二大学; 莱昂·贝拉尔中心; EPITA（法国高等计算机与技术学院））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出基于路径特征表示的多模态序列生存建模框架MultiSigBERT，结合电子健康记录的多模态数据与时间特性，在含2500余名患者的肿瘤队列上取得0.743的一致性指数，提升了生存预测性能。
AI 中文摘要
机器学习已成为现代医疗保健的重要组成部分，异构数据源的整合为改善临床决策提供了前所未有的机会。电子健康记录（EHR）包含互补信息，包括叙述性临床报告、数值测量和结构化变量，但大多数生存模型仍局限于单一模态，或未能利用患者轨迹的时间性质。我们提出MultiSigBERT，一种基于路径特征表示的肿瘤学多模态序列生存建模统一框架。其中，叙述性医疗报告（自由文本）通过提取和平均上下文词嵌入转换为句子嵌入；这些表示随后通过模态特定的主成分分析（PCA）压缩，并与结构化协变量连接形成联合时间轨迹，再使用特征变换（Signature transform）进行编码，该变换来自粗糙路径（Rough Paths）理论，是一种无需监督即可有效捕获跨模态高阶时间交互的工具；计算得到的特征最终作为高维特征纳入LASSO正则化Cox模型，以估计个体化风险评分。我们在来自Léon Bérard中心的真实世界肿瘤学队列上验证了MultiSigBERT流程的性能，该队列包含超过120,000份医疗报告和超过2,500名患者的结构化记录。该模型在独立测试集上的一致性指数为0.743（标准差0.029），证明了联合建模多模态时间动态与患者水平几何结构对生存预测的益处。
英文摘要
Machine learning has become an essential component of modern healthcare, where the integration of heterogeneous data sources offers unprecedented opportunities to improve clinical decision-making. Electronic Health Records (EHR) contain complementary information -- including narrative clinical reports, numerical measurements, and structured variables -- yet most survival models remain limited to a single modality or fail to exploit the temporal nature of patient trajectories. We propose MultiSigBERT, a unified framework for multimodal sequential survival modeling in oncology based on path signature representations. Here, narrative medical reports (free-text) are converted into sentence embeddings by extracting and averaging contextual word embeddings. These representations are then compressed via modality-specific PCA and concatenated with structured covariates to form joint temporal trajectories which are then encoded using the Signature transform, a tool from Rough Paths theory that efficiently captures higher-order temporal interactions across modalities without supervision needed. The computed Signature features are finally incorporated as high dimensional features into a LASSO-regularized Cox model to estimate individualized risk scores. The performance of our novel MultiSigBERT pipeline is illustrated on the analysis of a real-world oncology cohort from the Léon Bérard Center, comprising over 120,000 medical reports and structured records from more than 2,500 patients. The model achieves a concordance index of 0.743 (sd 0.029) on an independent test set, demonstrating the benefit of jointly modeling multimodal temporal dynamics together with patient-level geometric structure for survival prediction.
CommentsAccepted at ECML PKDD 2026, Applied Data Science Track
