CoSMoEs：紧凑稀疏混合专家模型
CoSMoEs: Compact Sparse Mixture of Experts
AI总结：
本文提出CoSMoEs框架，通过权重分解专家提升设备端MoE模型质量，并优化内存卸载以降低推理延迟，证明其在公平评估下优于FLOP对齐的稠密模型。
摘要：
稀疏混合专家（MoE）模型是流行的大规模基础架构，但在较小规模上研究不足。在此，我们展示了如何为设备端推理启用紧凑稀疏混合专家（CoSMoEs）。具体而言，我们应对了设备端的三个主要维度：质量、内存和延迟。在质量方面，我们展示了在公平评估（消除混淆因素）中，MoE架构在设备端规模上优于FLOP对齐的稠密模型。我们引入了权重分解专家，进一步提升了MoE模型的性能。关于模型内存和延迟，我们显著提升了模型卸载效率，进而降低了模型推理延迟。
英文摘要：
Sparse Mixture of Expert (MoE) models are popular foundational architectures at large scale, however, under-explored at smaller sizes. Here, we show how to enable Compact Sparse Mixture of Experts (CoSMoEs) for on-device inference. Specifically, we tackle the three main on-device dimensions: Quality, Memory and Latency. Along the quality axis, we show that in a fair evaluation (removing confounding factors) MoE architectures outperform FLOP-aligned dense models at on-device scale. We introduce weight-decomposed experts, further improving the MoE model performance. Regarding model memory and latency, we significantly improve model offloading efficiency and, in turn, reduce model inference latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
