LongMoE：基于轨迹感知的混合专家模型的纵向多模态学习
LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts
AI总结：
提出LongMoE框架，通过上下文感知插补、注意力标记化、轨迹感知编码和稀疏MoE路由，联合解决临床多模态学习中模态缺失和纵向动态两大挑战，在ADNI等数据集上验证了鲁棒性。
摘要：
多模态临床学习对于整合包括影像、文本和个性化健康记录在内的多样化患者数据日益重要。然而，它面临两个基本挑战：i) 模态缺失，即在一次患者就诊中任意子集的模态不可用；ii) 纵向动态，即观察结果的诊断意义取决于患者随时间演变的疾病轨迹。现有方法孤立地处理这些挑战：缺失模态框架将每次就诊视为独立的静态快照并丢弃时间上下文，而纵向模型通常假设模态完全可用并在系统性模态不完整时性能下降。我们提出LongMoE（纵向混合专家模型），这是一个统一框架，用于联合解决这两个挑战。LongMoE结合了上下文感知插补模块和注意力标记化模块，后者捕获不规则就诊序列中的频域时间模式，以及用于建模疾病进展的轨迹感知编码器和用于患者特定专家选择的上下文条件稀疏MoE路由。在ADNI、OASIS-3和MIMIC-IV上的实验表明，LongMoE在缺失或弱共时模态下提高了鲁棒性，并在全模态设置中保持竞争力，为纵向感知的多模态临床学习奠定了坚实基础。
英文摘要：
Multimodal clinical learning is increasingly important for integrating diverse patient data, including imaging, text, and personalised health records. However, it faces two fundamental challenges: i) modality missingness, where arbitrary subsets of modalities are unavailable at a given patient visit, ii) longitudinal dynamics, where the diagnostic significance of an observation depends on the patient's evolving disease trajectory over time. Existing methods address these challenges in isolation: missing-modality frameworks treat each visit as an independent static snapshot and discard temporal context, while longitudinal models often assume complete modality availability and degrade under systematic modality incompleteness. We propose LongMoE (Longitudinal Mixture-of-Experts), the unified framework to jointly address both challenges. LongMoE combines a context-aware imputation module with an attentional tokenization module that captures frequency-domain temporal patterns across irregular visit sequences, a trajectory-aware encoder for modeling disease progression, and context-conditioned Sparse MoE routing for patient-specific expert selection. Experiments on ADNI, OASIS-3, and MIMIC-IV show that LongMoE improves robustness under missing or weak contemporaneous modalities and remains competitive in full-modality settings, establishing a strong foundation for longitudinally-aware multimodal clinical learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
