角色专业化对混合专家模型解释忠实度重要吗？
Does Role Specialization Matter for Explanation Faithfulness in Mixture-of-Experts?
AI总结：
研究混合专家模型中专家角色分解对解释忠实度的影响，提出表示级去相关正则化减少专家间重叠，实验表明该方法能提升忠实度指标且不损害任务性能。
摘要：
混合专家（MoE）架构最近被扩展了基于角色的可解释性机制。这通常通过为单个专家组件分配语义角色来实现，例如在多模态设置中的协同、冗余和唯一性角色。然而，这种结构角色分解是否保持整体架构的解释忠实度在很大程度上仍未探索。我们假设专家间的表示重叠会削弱有效的角色分离并降低基于归因的忠实度，即使语义角色被明确定义。为解决这一限制，我们引入表示级去相关正则化，以显式减少潜在空间中专家间的相似性。通过表示去相关目标，我们通过最小化表示重叠来鼓励专家间更清晰的专业化。我们的实验表明，在多个多模态基准上，这种分离一致地提高了解释忠实度，通过全面性、充分性及其扰动曲线下面积（AOPC）摘要来衡量，同时保持了任务性能。我们进一步表明，这些改进不仅限于基于角色的架构，如可解释多模态交互感知MoE（I2MoE）。在标准稀疏MoE基线中也观察到类似趋势，表明表示级分离可能为增强MoE系统中的解释忠实度提供更通用的机制。总体而言，我们的发现表明，仅结构角色分解可能不足以保证忠实解释，而表示级分离有助于提高解释忠实度。为支持可重复性，源代码和补充材料可在 https://github.com/dut0817/FL-I2MoE_Decor 公开获取。
英文摘要：
Mixture-of-Experts (MoE) architectures have recently been extended with role-based mechanisms for interpretability. This is typically done by assigning semantic roles to individual expert components, for example roles like synergy, redundancy, and uniqueness in multimodal settings. However, whether such structural role decomposition preserves explanation faithfulness of the overall architecture remains largely underexplored. We hypothesize that inter-expert representation overlap weakens effective role separation and degrades attribution-based faithfulness, even when semantic roles are explicitly defined. To address this limitation, we introduce representation-level decorrelation regularization to explicitly reduce inter-expert similarity in latent space. Using representation decorrelation objectives, we encourage clearer specialization among experts by minimizing representation overlap. Our experiments show that across multiple multimodal benchmarks, this separation consistently improves explanation faithfulness, as measured by comprehensiveness, sufficiency, and their Area Over the Perturbation Curve (AOPC) summaries, while preserving task performance. We further show that these improvements are not limited to role-based architectures such as Interpretable Multimodal Interaction-aware MoE (I2MoE). Similar trends are observed in a standard sparse MoE baseline, suggesting that representation-level separation may provide a more general mechanism for enhancing explanation faithfulness in MoE systems. Overall, our findings suggest that structural role decomposition alone may be insufficient to guarantee faithful explanations and that representation-level separation helps improve explanation faithfulness. To support reproducibility, the source code and supplementary material are publicly available at https://github.com/dut0817/FL-I2MoE_Decor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
