从分数分布到负载均衡：即插即用的混合专家路由
From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing
AI总结：
针对混合专家模型推理时专家负载不均导致系统性能下降的问题，提出即插即用的推理时路由算法LASER，可适配门控分数分布动态调整路由策略，在不重训练的前提下提升负载均衡效果与推理性能且准确率损失极小。
摘要：
混合专家（Mixture-of-Experts, MoE）模型可通过学习得到的门控函数将每个token路由到部分专家，从而扩展参数容量。虽然条件路由降低了训练成本，但它将负担转移到了推理内存上：专家参数与激活值会占用内存，限制了单设备可承载的专家数量。在token路由过程中，部分专家会过载，而另一些则利用率不足。由于专家被映射到GPU上，这种不均衡会直接导致系统在延迟、吞吐量和成本方面的性能下降。我们提出了LASER，这是一种即插即用的推理时路由算法，可在保持准确率的同时实现负载均衡。LASER会根据门控分数分布的形态进行调整：当分数呈现明确的偏好时，它会将token路由到最强的专家；当分数更均匀时，它会扩大可行专家的范围，并将token路由到其中负载最低的专家。由于LASER仅依赖训练后模型的门控分数，它可以直接集成到现有的MoE推理流水线中，无需重新训练或微调。我们在Mixtral-8x7B和DeepSeek-MoE-16b-chat两个模型上，跨四个数据集（ARC-Easy、ARC-Challenge、MMLU和GSM8K）对LASER进行了评估。LASER改善了负载均衡，实现了更低的延迟和更高的吞吐量，同时准确率变化可忽略不计。
英文摘要：
Mixture-of-Experts (MoE) models can scale parameter capacity by routing each token to a subset of experts through a learned gate function. While conditional routing reduces training costs, it shifts the burden on inference memory: expert parameters and activations consume memory, limiting the number of experts per device. As tokens are routed, some experts become overloaded while others are underutilized. Because experts are mapped to GPUs, this imbalance translates directly into degraded system performance in terms of latency, throughput, and cost. We present LASER, a plug-and-play, inference-time routing algorithm that balances load while preserving accuracy. LASER adapts to the shape of the gate's score distribution. When scores provide a clear preference, it routes to the strongest experts; when scores are more uniform, it broadens the set of viable experts and routes to the least-loaded among them. Because LASER relies only on gate scores from a trained model, it integrates directly into existing MoE inference pipelines without retraining or finetuning. We evaluate LASER on Mixtral-8x7B and DeepSeek-MoE-16b-chat across four datasets (ARC-Easy, ARC-Challenge, MMLU, and GSM8K). LASER improves load balancing, translating into lower latency and higher throughput, while keeping the accuracy changes negligible.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
