arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.19503cs.DC
收藏

ReaLB:多模态MoE推理的实时负载均衡

ReaLB: Real-Time Load Balancing for Multimodal MoE Inference

The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))

Yingping Wang, Yi Wu, Xiangyu Wu, Junwei Cui, Weilin Cai, Zhijiang Guo, Jiayi Huang

AI总结:

ReaLB通过实时调整MoE专家计算精度,提升多模态模型推理效率,减少负载不平衡,实现1.10-1.32倍的端到端加速,同时保持1%以内的精度损失。

摘要:

ReaLB通过实时调整MoE专家计算精度,提升多模态模型推理效率,减少负载不平衡,实现1.10-1.32倍的端到端加速,同时保持1%以内的精度损失。

英文摘要:

Mixture-of-Experts (MoE) architectures are widely used in modern large language models and multimodal models. However, inference efficiency is often limited by highly dynamic and skewed expert workloads across different modalities. During the prefill stage with large batch sizes, vision tokens frequently dominate the input sequences. Under expert parallelism (EP), this leads to severe load imbalance, where a subset of devices becomes overloaded, reducing overall system throughput. We propose ReaLB, a real-time load balancing method for multimodal MoE (MMoE) inference that introduces zero scheduling overhead. ReaLB dynamically adjusts the computation precision of MoE experts at runtime on a per-EP-rank basis. For ranks dominated by vision-heavy experts, ReaLB assigns lower-precision computation to improve execution efficiency by exploiting FP4 Tensor Cores. ReaLB does not require redundant experts or additional memory allocation. Instead, it performs layer-wise expert precision transformation on the fly and hides the associated overhead within the dispatch phase before MoE computation. Experiments on representative MMoE models show that ReaLB achieves 1.10$\times$-1.32$\times$ end-to-end speedup while limiting average accuracy degradation to within 1%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑