重新思考稀疏MoE中的门控机制：通过置信度引导的门来处理任意模态输入
Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gate
AI总结：
本文提出ConfSMoE，通过两阶段填补模块解决稀疏MoE中缺失模态问题，引入置信度引导门控机制，缓解专家崩溃问题，提升模型鲁棒性。
摘要：
有效管理缺失模态是现实多模态学习场景中的基本挑战，数据不完整常源于系统性收集错误或传感器故障。稀疏混合专家（SMoE）架构有潜力自然处理多模态数据，但现有SMoE方法缺乏处理缺失模态的能力，导致性能下降和泛化能力差。本文提出ConfSMoE，通过两阶段填补模块处理SMoE架构中的缺失模态问题，结合专家意见和理论分析揭示专家崩溃的洞察。受理论分析启发，ConfSMoE提出新颖的专家门控机制，通过分离softmax路由分数到任务置信度分数，自然缓解专家崩溃而不引入额外负载平衡损失函数。我们展示专家崩溃的洞察与Gaussian和Laplacian门控机制一致。所提方法在四个不同现实数据集上进行评估，通过三种不同实验设置进行综合分析，探讨ConfSMoE对缺失模态的抗性及所提门控机制的影响。
英文摘要：
Effectively managing missing modalities is a fundamental challenge in real-world multimodal learning scenarios, where data incompleteness often results from systematic collection errors or sensor failures. Sparse Mixture-of-Experts (SMoE) architectures have the potential to naturally handle multimodal data, with individual experts specializing in different modalities. However, existing SMoE approach often lacks proper ability to handle missing modality, leading to performance degradation and poor generalization in real-world applications. We propose ConfSMoE to introduce a two-stage imputation module to handle the missing modality problem for the SMoE architecture by taking the opinion of experts and reveal the insight of expert collapse from theoretical analysis with strong empirical evidence. Inspired by our theoretical analysis, ConfSMoE propose a novel expert gating mechanism by detaching the softmax routing score to task confidence score w.r.t ground truth signal. This naturally relieves expert collapse without introducing additional load balance loss function. We show that the insights of expert collapse aligns with other gating mechanism such as Gaussian and Laplacian gate. The proposed method is evaluated on four different real world dataset with three distinct experiment settings to conduct comprehensive analysis of ConfSMoE on resistance to missing modality and the impacts of proposed gating mechanism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
