arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.08958cs.LGcs.AI
收藏

迈向超大规模Transformer的下一代训练后量化

Towards Next-Level Post-Training Quantization of Hyper-Scale Transformers

Samsung Research(三星研究院)

Junhan Kim, Chungman Lee, Eulrang Cho, Kyungphil Park, Ho-young Kim, Joonyoung Kim, Yongkweon Jeon

AI总结:

本文提出一种名为aespa的新型训练后量化算法,通过逐层量化与注意力级重建相结合,在保证效率的同时考虑跨层依赖,实现对超大规模Transformer模型的准确高效量化。

摘要:

随着生成式AI模型复杂度的不断增加,训练后量化(PTQ)已成为在移动设备和电视等边缘设备上部署超大规模模型的有前景的解决方案。然而,现有的PTQ方案消耗大量时间和资源,在需要频繁模型更新和多次超参数调整的真实场景中可能成为瓶颈。作为一种经济高效的替代方案,免学习PTQ方案已被提出。然而,其性能有些受限,因为它们无法考虑注意力模块内的层间依赖性,而这是Transformer的一个重要特征。因此,在本文中,我们提出了一种新颖的PTQ算法,以平衡准确性和效率。所提出的名为aespa的算法的关键思想是逐层执行量化以提高效率,同时以注意力级重建为目标以考虑跨层依赖性。通过对各种语言模型的广泛实验和复杂度分析,我们证明了aespa在量化Transformer模型方面既准确又高效。

英文摘要:

With the increasing complexity of generative AI models, post-training quantization (PTQ) has emerged as a promising solution for deploying hyper-scale models on edge devices such as mobile and TVs. Existing PTQ schemes, however, consume considerable time and resources, which could be a bottleneck in real situations where frequent model updates and multiple hyperparameter tunings are required. As a cost-effective alternative, learning-free PTQ schemes have been proposed. However, the performance is somewhat limited because they cannot consider the inter-layer dependency within the attention module, which is a significant feature of Transformers. In this paper, we thus propose a novel PTQ algorithm that balances accuracy and efficiency. The key idea of the proposed algorithm called aespa is to perform quantization layer-wise for efficiency while targeting attention-wise reconstruction to consider the cross-layer dependency. Through extensive experiments on various language models and complexity analysis, we demonstrate that aespa is accurate and efficient in quantizing Transformer models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑