RadioPathomics：用于非小细胞肺癌自适应放疗的多模态学习
RadioPathomics: Multimodal Learning in Non-Small Cell Lung Cancer for Adaptive Radiotherapy
AI总结：
针对非小细胞肺癌放疗结局预测，提出结合放射组学、病理组学与临床数据的多模态晚期融合方法，经33例患者队列验证AUC达90.9%，优于单模态及早期融合等方法，助力精准放疗。
摘要：
当前的癌症治疗实践会采集多模态数据，例如放射影像、组织病理学切片、基因组学数据以及临床数据。这些数据源各自的重要性推动了近期放射组学（radiomics）和病理组学（pathomics）的兴起，即从常规采集的放射影像和组织病理学图像中提取定量特征，再通过人工智能算法预测临床结局或指导临床决策。然而，如何将这些数据整合到单个多模态框架中仍是一个待解决的问题。因此，本研究开发了一种多模态晚期融合方法，结合从放射组学、病理组学及临床数据中计算得到的手工特征，用于预测非小细胞肺癌患者的放疗结局。在此框架下，我们研究了八种不同的晚期融合规则（即乘积、最大值、最小值、均值、决策模板、Dempster-Shafer、多数投票以及置信度规则），以及两种利用计算机断层扫描图像和全切片扫描提供的丰富信息的患者级聚合规则。在包含33名患者的内部队列上开展的留一患者交叉验证实验显示，所提出的多模态范式的AUC达到90.9%，优于所有单模态方法，表明数据整合能够推动精准医学发展。作为另一项贡献，我们还将手工特征表示与深度网络自动计算的特征进行了对比，同时将晚期融合范式与另一种主流多模态方法早期融合进行了比较。两类对比实验均表明，所提出的多模态方法取得了最优结果。
英文摘要：
The current cancer treatment practice collects multimodal data, such as radiology images, histopathology slides, genomics and clinical data. The importance of these data sources taken individually has fostered the recent raise of radiomics and pathomics, i.e. the extraction of quantitative features from radiology and histopathology images routinely collected to predict clinical outcomes or to guide clinical decisions using artificial intelligence algorithms. Nevertheless, how to combine them into a single multimodal framework is still an open issue. In this work we therefore develop a multimodal late fusion approach that combines hand-crafted features computed from radiomics, pathomics and clinical data to predict radiation therapy treatment outcomes for non-small-cell lung cancer patients. Within this context, we investigate eight different late fusion rules (i.e. product, maximum, minimum, mean, decision template, Dempster-Shafer, majority voting, and confidence rule) and two patient-wise aggregation rules leveraging the richness of information given by computer tomography images and whole-slide scans. The experiments in leave-one-patient-out cross-validation on an in-house cohort of 33 patients show that the proposed multimodal paradigm with an AUC equal to $90.9\%$ outperforms each unimodal approach, suggesting that data integration can advance precision medicine. As a further contribution, we also compare the hand-crafted representations with features automatically computed by deep networks, and the late fusion paradigm with early fusion, another popular multimodal approach. In both cases, the experiments show that the proposed multimodal approach provides the best results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
