AI 中文总结
GyRot通过算法-硬件协同设计，结合旋转与细粒度组量化，在LLaMA系列模型4位量化推理中实现先进精度，同时提升加速比与能效，为LLM部署提供可行方案。
AI 中文摘要
低位量化对高效的大语言模型（LLM）推理至关重要，旋转和细粒度组量化均展现出各自的应用潜力。然而，二者的组合常因旋转的全局特性与组缩放的局部行为不匹配，导致精度下降或硬件开销。我们提出GyRot，一种量化框架与硬件加速器，通过算法-硬件协同设计弥合该差距。GyRot引入粗旋转、细分组（CoRFiG）和谐波对齐置换（HAP），实现旋转与组量化的协同集成，提升量化能力同时放宽缩放因子精度要求。为进一步降低硬件成本，我们重构非对称量化并引入零点舍入策略，实现完全整数反量化。在基于INT4的张量处理单元（PE）架构上实现的GyRot，在LLaMA系列模型中达到先进的4位精度，同时较基线LLM加速器实现最高3.4倍的加速比和3.6倍的能效提升。这些结果验证了GyRot在可扩展且高能效的LLM部署中的实际有效性。
英文摘要
Low-bit quantization is essential for efficient LLM inference, and both rotation and fine-grained group quantization have shown individual promise. However, their combination often leads to accuracy degradation or hardware overhead due to a mismatch between the global nature of rotation and the localized behavior of group scaling. We propose GyRot, a quantization framework and hardware accelerator that bridges this gap through algorithm-hardware co-design. GyRot introduces Coarse Rotation, Fine Grouping (CoRFiG) and Harmonic-Aligned Permutation (HAP) to enable cooperative integration of rotation and group quantization, enhancing quantizability while relaxing scaling factor precision. To further reduce hardware cost, we reformulate asymmetric quantization and introduce a zero-point rounding strategy that enables fully integer dequantization. Implemented on an INT4-based tensor PE architecture, GyRot achieves state-of-the-art 4-bit accuracy across LLaMA-family models, while delivering up to 3.4x speedup and 3.6x energy efficiency over baseline LLM accelerators. These results validate GyRot's practical effectiveness for scalable and energy-efficient LLM deployment.
Comments15 pages, 12 figures. Published in 2026 IEEE International Symposium on High-Performance Computer Architecture (HPCA), Sydney, Australia, pp. 1-15, DOI: 10.1109/HPCA68181.2026.11408453
Journal refProc. 2026 IEEE Int. Symp. High-Performance Computer Architecture (HPCA), 2026, pp. 1-15
DOI：10.1109/HPCA68181.2026.11408453
