TileQ: 一种高效的混合专家低秩量化方法基于二维铺砖
TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling
AI总结：
TileQ通过二维铺砖结构低秩量化方法，在不进行微调的情况下实现混合专家模型的高效量化，显著降低内存使用和推理延迟，同时保持最先进的准确性。
摘要：
混合专家（MoE）模型通过稀疏激活专用专家实现卓越性能，但专家的大量参数对部署构成重大挑战。尽管低秩量化为压缩MoE模型提供了有前途的途径，但现有方法仍导致不可忽视的内存开销和推理延迟。为了解决这些限制，我们提出TileQ，一种无需微调的后训练量化（PTQ）方法，采用二维铺砖结构低秩量化，在输入和输出维度的MoE专家之间共享低秩因子。此外，我们引入了一种高效的推理技术，将多个低秩专家计算融合为单次操作，显著提高了硬件利用率。实验表明，TileQ将额外内存使用减少高达10倍，并将推理延迟降低到约5%，同时保持最先进的准确性。
英文摘要：
Mixture-of-Experts (MoE) models achieve remarkable performance by sparsely activating specialized experts, yet their massive parameters in experts pose significant challenges for deployment. While low-rank quantization offers a promising route to compress MoE models, existing methods still incur nonnegligible memory overhead and inference latency. To address these limitations, we propose \textsc{TileQ}, a fine-tuning-free post-training quantization (PTQ) method that employs 2D-tiling structured low-rank quantization to share low-rank factors across both input and output dimensions of MoE experts. Furthermore, we introduce an efficient inference technique for \textsc{TileQ} that fuses multiple low-rank expert computations into a single-pass operation, significantly improving hardware utilization. Experiments show that \textsc{TileQ} cuts down additional memory usage up to 10$\times$ and reduces inference latency to $\sim$5\% while preserving state-of-the-art accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
