MicroScopiQ：通过异常值感知的微缩放量化加速基础模型
MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization
AI总结：
本文提出MicroScopiQ，一种结合剪枝与异常值感知量化的协同设计技术，通过剪枝最不重要的权重来分配异常值比特，在保证高精度的同时实现了高硬件效率，推理速度最高提升3倍。
摘要：
由于被称为异常值的大幅度值的出现，基础模型（FMs）的量化比传统DNNs更具挑战性。现有的异常值感知算法-架构协同设计技术要么使用混合精度，以高精度保留异常值但损害硬件效率；要么以相同精度量化内部值和异常值，以牺牲准确度为代价提高硬件效率。为了解决这种互斥性，我们提出了MicroScopiQ，一种利用剪枝来补充异常值量化的新型协同设计技术。MicroScopiQ以较高精度保留异常值，同时剪枝一部分最不重要的权重以分配额外的异常值比特；从而确保高准确度、对齐的内存和硬件效率。我们设计了一种高吞吐量、低开销的加速器架构，由多精度INT处理元素和一个名为ReCoN的片上网络组成，该网络有效地抽象了支持高精度异常值的复杂性。此外，与现有技术不同，MicroScopiQ不假设异常值权重具有任何局部性，从而能够适用于广泛的FMs。在不同量化设置下的大量实验表明，MicroScopiQ实现了最先进的量化准确度，同时与现有方案相比，推理速度最高提升3倍，能耗降低2倍。代码可在：https://github.com/georgia-tech-synergy-lab/MicroScopiQ-LLM-Quantization 获取
英文摘要：
Quantization of foundational models (FMs) is significantly more challenging than traditional DNNs due to the emergence of large magnitude values called outliers. Existing outlier-aware algorithm-architecture co-design techniques either use mixed-precision, retaining outliers at high precision but compromise hardware efficiency, or quantize inliers and outliers at the same precision, improving hardware efficiency at the cost of accuracy. To address this mutual exclusivity, we propose MicroScopiQ, a novel co-design technique that leverages pruning to complement outlier-aware quantization. MicroScopiQ retains outliers at higher precision while pruning a certain fraction of least important weights to distribute the additional outlier bits; ensuring high accuracy, aligned memory and hardware efficiency. We design a high-throughput, low overhead accelerator architecture composed of multi-precision INT processing elements and a network-on-chip called ReCoN that efficiently abstracts the complexity of supporting high-precision outliers. Additionally, unlike prior techniques, MicroScopiQ does not assume any locality of outlier weights, enabling applicability to a broad range of FMs. Extensive experiments across diverse quantization settings demonstrate that MicroScopiQ achieves state-of-the-art quantization accuracy, while delivering up to 3x faster inference and 2x lower energy consumption compared to existing alternatives. Code is available at: https://github.com/georgia-tech-synergy-lab/MicroScopiQ-LLM-Quantization
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
