ISQuant:将squant应用于实际部署
ISQuant: apply squant to the real deployment
AI总结:
针对伪量化与实际部署的差距,提出ISQuant方法,快速部署8位模型,无需训练数据,实验验证效果可接受。
摘要:
深度神经网络的模型量化技术已引起广泛关注,并被证明在压缩模型大小、降低计算成本和加速推理方面非常有用。许多研究者采用伪量化来分析和训练量化过程。然而,伪量化并非部署的最终形式,学术环境与实际部署之间存在差距。此外,引入尺度和零点带来的额外计算使部署成为一项具有挑战性的任务。在本研究中,我们首先分析了为何使用量化和反量化的组合来训练模型,并得出结论:由于权重梯度的消失以及伪量化与实际量化之间的近似能力,伪量化研究是合理的。其次,我们提出了ISQuant作为部署8位模型的解决方案。ISQuant对于大多数8位模型而言快速且易于使用,所需参数更少,计算量更小。ISQuant还继承了SQuant的优点,如不需要训练数据,并且在一级量化时速度非常快。最后,我们进行了一些实验,发现结果是可以接受的。我们的代码可在https://github.com/获取。
英文摘要:
The model quantization technique of deep neural networks has garnered significant attention and has proven to be highly useful in compressing model size, reducing computation costs, and accelerating inference. Many researchers employ fake quantization for analyzing or training the quantization process. However, fake quantization is not the final form for deployment, and there exists a gap between the academic setting and real-world deployment. Additionally, the inclusion of additional computation with scale and zero-point makes deployment a challenging task. In this study, we first analyze why the combination of quantization and dequantization is used to train the model and draw the conclusion that fake quantization research is reasonable due to the disappearance of weight gradients and the ability to approximate between fake and real quantization. Secondly, we propose ISQuant as a solution for deploying 8-bit models. ISQuant is fast and easy to use for most 8-bit models, requiring fewer parameters and less computation. ISQuant also inherits the advantages of SQuant, such as not requiring training data and being very fast at the first level of quantization. Finally We conduct some experiments and found the results is acceptable.our code is available at https://github.com/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。