DBES: 一种用于评估大规模MoE模型专家专业化程度的系统性基准和度量套件
DBES: A Systematic Benchmark and Metric Suite for Evaluating Expert Specialization in Large-Scale MoEs
AI总结：
本文提出DBES系统性基准和度量套件，通过多领域基准和五个理论基础的度量指标，评估MoE模型中的专家专业化程度，并验证这些度量指标在领域特定后训练中的可操作性，实现了显著的性能提升。
摘要：
MoE模型中的专家专业化仍缺乏深入理解，传统评估将架构负载均衡与功能专业化混淆。我们引入DBES，一种综合的诊断框架，结合多领域基准和五个理论基础的度量指标：路由专业化、归一化有效秩、领域隔离、路由刚度分数和n-gram专家度量。关键发现显示不同模型展现出不同的专业化范式：Qwen系列表现出模块化专业化，具有高领域隔离，而DeepSeek和GLM采用分布式协作。然而，我们强调专业化是诊断维度，必要但不充分用于下游性能。最重要的是，干预证据验证了这些度量指标的可操作性：通过使用DBES在领域特定后训练中识别高专业化专家路径，我们仅使用15%的原始训练资源，在专业化领域实现了66%至94.48%的性能提升，证明这些诊断工具可以转化为具体的优化算子。本文提供了首个系统性的方法，用于独立于准确度指标评估专家专业化，为下一代MoE系统的设计和后训练优化提供了关键见解。
英文摘要：
Expert specialization in Mixture-of-Experts (MoE) models remains poorly understood, with traditional evaluations conflating architectural load-balancing with functional specialization. We introduce DBES, a comprehensive diagnostic framework combining a multi-domain benchmark with five theoretically grounded metrics: Routing Specialization, Normalized Effective Rank, Domain Isolation, Routing Stiffness Score, and N-gram Expertise measures. Critical findings demonstrate distinct specialization paradigms across models: Qwen-series exhibit modular specialization with high domain isolation, while DeepSeek and GLM employ distributed collaboration. However, we emphasize that specialization is a diagnostic dimension, necessary but not sufficient for downstream performance. Most crucially, interventional evidence validates the actionability of these metrics: by using DBES to identify high-specialization expert paths during domain-specific post-training, we achieved 66% to 94.48% improvement in specialized domains with only 15% of original training resources, demonstrating that these diagnostic tools can be converted into concrete optimization operators. This work provides the first systematic methodology for evaluating expert specialization independently of accuracy metrics, offering crucial insights for the design and post-training optimization of next-generation MoE systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
