带宽高效自适应混合专家 via 低秩补偿
Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation
AI总结：
本文提出了一种带宽高效自适应混合专家方法，通过低秩补偿实现精度恢复，提升带宽-准确性权衡和吞吐量。
摘要：
混合专家（MoE）模型通过稀疏激活来扩展容量，但会带来内存和带宽的压力。卸载通过按需获取专家来减轻GPU内存，但令牌级路由导致不规则传输，使推理I/O受限。静态均匀量化减少流量但通过强烈压缩会降低准确性，因为它忽略了专家异质性。我们提出了带宽高效自适应混合专家 via 低秩补偿，该方法通过预先计算的低秩补偿器进行路由器引导的精度恢复。在推理时，我们的方法传输紧凑的低秩因子，每个令牌选择Top-n（n<k）专家，并对它们应用补偿，保持其他专家的低比特。与GPU和GPU-NDP系统的卸载相结合，我们的方法实现了更优的带宽-准确性权衡和改进的吞吐量。
英文摘要：
Mixture-of-Experts (MoE) models scale capacity via sparse activation but stress memory and bandwidth. Offloading alleviates GPU memory by fetching experts on demand, yet token-level routing causes irregular transfers that make inference I/O-bound. Static uniform quantization reduces traffic but degrades accuracy under aggressive compression by ignoring expert heterogeneity. We present Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation, which performs router-guided precision restoration using precomputed low-rank compensators. At inference time, our method transfers compact low-rank factors with Top-n (n<k) experts per token and applies compensation to them, keeping others low-bit. Integrated with offloading on GPU and GPU-NDP systems, our method delivers a superior bandwidth-accuracy trade-off and improved throughput.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
