激发：专家的动量

AI总结：

Excitation是一种无需额外参数的优化框架，通过动态调节专家更新提升稀疏架构模型的训练效率和性能。

摘要：

我们提出了Excitation，一种新的优化框架，旨在加速稀疏架构（如混合专家MoEs）中的学习。与传统优化器将所有参数统一处理不同，Excitation利用批次层面的专家利用率动态调节更新。它引入了竞争更新动态，放大对高利用率专家的更新，并可选择性抑制低利用率的专家，从而有效提升路由的专业性。值得注意的是，我们发现了深度MoEs中的

英文摘要：

We propose Excitation, a novel optimization framework designed to accelerate learning in sparse architectures such as Mixture-of-Experts (MoEs). Unlike traditional optimizers that treat all parameters uniformly, Excitation dynamically modulates updates using batch-level expert utilization. It introduces a competitive update dynamic that amplifies updates to highly-utilized experts and can selectively suppress low-utilization ones, effectively sharpening routing specialization. Notably, we identify a phenomenon of "structural confusion" in deep MoEs, where standard optimizers fail to establish functional signal paths; Excitation acts as a specialization catalyst, "rescuing" these models and enabling stable training where baselines remain trapped. Excitation is optimizer-, domain-, and model-agnostic, requires minimal integration effort, and introduces neither additional per-parameter optimizer state nor learnable parameters, making it highly viable for memory-constrained settings. Across language and vision tasks, Excitation consistently improves convergence speed and final performance in MoE models, indicating that active update modulation is a key mechanism for effective conditional computation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。