MoNDE：面向大规模稀疏模型的近数据专家混合
MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models
AI总结：
MoNDE提出近数据计算方案，仅传输热专家至GPU并在内存中计算冷专家，减少参数移动，实现高效MoE推理并显著加速。
摘要：
混合专家（MoE）大语言模型（LLM）的内存需求往往超过GPU内存容量，导致在专家计算时需要将昂贵的参数从次级内存移动到GPU。在本工作中，我们提出了近数据专家混合（MoNDE），一种近数据计算解决方案，可高效支持MoE LLM推理。MoNDE通过仅将“热”专家传输到GPU，而在主机内存设备内计算剩余的“冷”专家，从而减少MoE参数移动量。通过用小型激活的传输替代大规模专家参数的传输，MoNDE实现了远更高效的通信MoE推理，从而在编码器和解码器操作上相较于现有的参数卸载框架获得了显著的加速。
英文摘要：
Mixture-of-Experts (MoE) large language models (LLM) have memory requirements that often exceed the GPU memory capacity, requiring costly parameter movement from secondary memories to the GPU for expert computation. In this work, we present Mixture of Near-Data Experts (MoNDE), a near-data computing solution that efficiently enables MoE LLM inference. MoNDE reduces the volume of MoE parameter movement by transferring only the $\textit{hot}$ experts to the GPU, while computing the remaining $\textit{cold}$ experts inside the host memory device. By replacing the transfers of massive expert parameters with the ones of small activations, MoNDE enables far more communication-efficient MoE inference, thereby resulting in substantial speedups over the existing parameter offloading frameworks for both encoder and decoder operations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
