arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.07471cs.LGcs.AI
收藏

SQuant:基于对角Hessian近似的动态无数据Quantization

SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation

Shanghai Jiao Tong University(上海交通大学);Shanghai Qi Zhi Institute(上海齐智研究所);Microsoft Research(微软研究院);DAMO Academy, Alibaba Group(阿里巴巴达摩院);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院);University of Rochester(罗切斯特大学)

Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, Minyi Guo

摘要:

深度神经网络(DNN)的量化已被证明可有效压缩和加速DNN模型。无数据量化(DFQ)是一种在隐私敏感和机密场景下不需要原始数据集的有前途的方法。然而,当前的DFQ解决方案降低了精度,需要合成数据来校准网络,并且耗时且成本高。本文提出了一种具有亚秒级量化时间的动态DFQ框架,称为SQuant,它可以在仅推理的设备上以较低的计算和内存要求来对网络进行建模。通过对DNN任务损失的二阶信息的理论分析,我们将基于Hessian的优化目标分解和近似为三个对角子项,这些子项具有对应于权重张量的三个维度的不同区域:元素方向,内核方向和输出通道方向。然后,我们逐步组成子项,并在离散域中提出一种新的无数据优化目标,最小化误差的约束绝对和(或简称CASE),令人惊讶的是,它不需要任何数据集,甚至不知道网络架构。我们还设计了一个有效的算法,没有反向传播,以进一步降低目标求解器的计算复杂度。最后,在没有微调和合成数据集的情况下,SQuant将无数据量化过程加速到亚秒级,与现有的无数据后训练量化工作相比,精度提高了>30%,评估模型采用4位量化。我们在https://github.com/clevercool/SQuant上开源了SQuant框架。

英文摘要:

Quantization of deep neural networks (DNN) has been proven effective for compressing and accelerating DNN models. Data-free quantization (DFQ) is a promising approach without the original datasets under privacy-sensitive and confidential scenarios. However, current DFQ solutions degrade accuracy, need synthetic data to calibrate networks, and are time-consuming and costly. This paper proposes an on-the-fly DFQ framework with sub-second quantization time, called SQuant, which can quantize networks on inference-only devices with low computation and memory requirements. With the theoretical analysis of the second-order information of DNN task loss, we decompose and approximate the Hessian-based optimization objective into three diagonal sub-items, which have different areas corresponding to three dimensions of weight tensor: element-wise, kernel-wise, and output channel-wise. Then, we progressively compose sub-items and propose a novel data-free optimization objective in the discrete domain, minimizing Constrained Absolute Sum of Error (or CASE in short), which surprisingly does not need any dataset and is even not aware of network architecture. We also design an efficient algorithm without back-propagation to further reduce the computation complexity of the objective solver. Finally, without fine-tuning and synthetic datasets, SQuant accelerates the data-free quantization process to a sub-second level with >30% accuracy improvement over the existing data-free post-training quantization works, with the evaluated models under 4-bit quantization. We have open-sourced the SQuant framework at https://github.com/clevercool/SQuant.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑