发表机构
Friedrich-Alexander-Universität; Klinikum Nürnberg, Paracelsus Medical University; Ludwig-Maximilians-Universität München; National Institute of Mental Health and Neurosciences (NIMHANS)（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学; 帕拉塞尔苏斯医科大学纽伦堡医院; 慕尼黑大学; 国家精神卫生与神经科学研究院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出PANDA框架，可在推理阶段无辅助输入时将不完整辅助模态的信息迁移至主模态模型，在阿尔茨海默病分类与TCGA生存预测任务中均取得优于基线的性能。
AI 中文摘要
多模态医疗预测常面临配对不完整的问题：仅部分受试者（或无受试者）拥有具备互补信号的辅助模态，且部署时无法假设存在这些辅助模态。本文提出PANDA（Prototype Anchored Data Alignment，原型锚定数据对齐），这是一种两阶段框架，可在推理阶段无辅助输入的情况下将辅助信息迁移至多模态模型。第一阶段从配对子集学习共享嵌入，并从辅助模态中估计类别原型；第二阶段使用交叉熵加对齐冻结原型的方式，在所有受试者上训练主模态编码器。由于监督信号定义在类别原型层面，PANDA可适配任意配对率，包括零受试者重叠。我们在两项应用中评估PANDA：在含1021名受试者的多扫描仪ADNI队列中，以三种不同配对率的辅助模态（表格评分配对率44.8%、FDG-PET配对率18.7%、外部书写运动学配对率0%重叠）执行AD/CN分类，与同骨干的仅MRI基线相比，PANDA的AUC达0.868±0.020（提升7.9个百分点），并将1.5T CN假阳性降低24.3个百分点；在可完全训练的Conv5-FC3骨干上，其AUC达0.893（整体最优）。配对率消融实验显示，联合锚点在75%至5%配对率范围内仍处于种子噪声内。在以RNA-seq为辅助数据的TCGA-Lung全切片图像生存预测中，PANDA较仅WSI方法提升2年OS（AUC提升3.5个百分点）与Cox PH（C指数提升9.0分），且优于全融合训练（其表现弱于仅WSI方法），同时推理阶段无需RNA数据；该较小队列的宽置信区间使增益未达常规显著性水平。总体而言，PANDA提供了一种面向部署的机制，用于利用不完整辅助模态提升主模态预测性能。
英文摘要
Multimodal medical prediction often faces incomplete pairing: auxiliary modalities with complementary signal are available for only a subset of subjects (or none) and cannot be assumed at deployment. We introduce PANDA (Prototype Anchored Data Alignment), a two-stage framework that transfers auxiliary information to a primary-modality model without auxiliary inputs at inference. Stage 1 learns a shared embedding from the paired subset and estimates class prototypes from auxiliary modalities; Stage 2 trains the primary encoder on all subjects using cross-entropy plus alignment to the frozen prototypes. Because supervision is defined at the class-prototype level, PANDA accommodates arbitrary pairing rates, including zero subject overlap. We evaluate PANDA on two applications. On a 1,021-subject multi-scanner ADNI cohort, we perform AD/CN classification with three auxiliary modalities at distinct pairing rates: tabular scores (44.8%), FDG-PET (18.7%), and external handwriting kinematics (0% overlap). Relative to the same-backbone MRI-only baseline, PANDA attains AUC 0.868 +-0.020 (+7.9pp) and reduces 1.5T CN false positives by 24.3pp; on a fully trainable Conv5-FC3 backbone it reaches AUC 0.893 (best overall). A pairing-rate ablation shows that the joint anchor remains within seed noise from 75% to 5% pairing. On TCGA-Lung survival prediction from whole-slide images with RNA-seq as auxiliary data, PANDA improves over WSI-only on 2-year OS (AUC +3.5pp) and Cox PH (C-index +9.0pts) and outperforms full-fusion training, which underperforms WSI-only, while requiring no RNA at inference; wide confidence intervals on this smaller cohort keep the gains below conventional significance. Overall, PANDA provides a deployment-oriented mechanism for leveraging incomplete auxiliary modalities to improve primary-modality prediction.
