TWIN-MoE：用于不完整多模态学习的两阶段专家内插补网络
TWIN-MoE: Two-stage Intra-Expert Imputation Network for Incomplete Multimodal Learning
AI总结：
本文提出TWIN-MoE框架，针对不完整多模态学习，通过两阶段专家内插补方法，在四类缺失模态基准上实现最优性能且参数更少。
摘要：
近期的混合专家（Mixture-of-Experts, MoE）方法处理缺失模态的方式有两种：一是将每种缺失模式路由至专用专家，但随着模态数量增加，该方法扩展性差；二是从多个专家贡献的表示中对缺失模态进行插补，但需要昂贵的检索机制。本文表明，路由会诱导出专家条件化的表示邻域，并推导了由其局部特征离散度控制的插补误差界。当这些专家条件化邻域比全局表示空间更紧凑时，专家内聚合比无约束的全局聚合具有更紧的最坏情况重构界。基于此分析，本文提出了TWIN，一种两阶段专家内插补框架：预插补在编码器嵌入空间生成上下文条件估计，并使用观测到的先进先出（FIFO）队列中维护的模态特定统计量对其进行径向软裁剪，确保预插补模态保持在目标模态空间的经验校准区域内；后插补在MoE之后，通过聚合同一样本内专家条件化的观测表示来细化缺失表示。在自然、随机和非对称丢弃协议下的四个缺失模态基准测试中，TWIN在达到最先进性能的同时，参数数量远少于基于MoE的插补对比方法。
英文摘要：
Recent Mixture-of-Experts (MoE) approaches address missing modalities either by routing each missingness pattern to a dedicated expert, which scales poorly as the number of modalities grows, or by imputing the missing modality from representations contributed by multiple experts, which requires a costly retrieval mechanism. We show that routing induces expert-conditioned representation neighborhoods and derive an imputation-error bound controlled by their local feature dispersion. When these expert-conditioned neighborhoods are more compact than the global representation space, intra-expert aggregation admits a tighter worst-case reconstruction bound than unconstrained global aggregation. Guided by this analysis, we propose TWIN, a two-stage intra-expert imputation framework. Pre-imputation generates a context-conditioned estimate in the encoder embedding space and radially soft-clips it using modality-specific statistics maintained in an observed FIFO queue, this ensures that the pre-imputed modality stays in empirically calibrated region of the target modality space. Post-imputation refines the missing representation after MoE by aggregating expert-conditioned observed representations within the same sample. Across four missing-modality benchmarks under natural, random, and asymmetric dropout protocols, TWIN achieves state-of-the-art performance while using substantially fewer parameters than competing MoE-based imputation methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
