arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.13716cs.CV
收藏

使用逆合成数据进行有效推理的混合和非均匀quantization方法

Hybrid and Non-Uniform quantization methods using retro synthesis data for efficient inference

Cranberry-Lemon University(蔓越莓柠檬大学);University of the Witwatersrand(威特沃特斯兰德大学)

Tej pratap GVSL, Raja Kumar

摘要:

现有的量化感知训练方法试图通过利用训练数据来补偿量化损失,像大多数训练后量化方法一样,并且也是耗时的。这两种方法对于隐私约束应用都不是有效的,因为它们与训练数据紧密耦合。相比之下,本文提出了一种数据独立的后训练量化方案,消除了对训练数据的需要。这是通过从FP 32模型层统计生成人造数据集(以下称为逆合成数据)并进一步将其用于量化来实现的。这种方法优于最先进的方法,包括但不限于ZeroQ和DFQ,在ImageNet和CIFAR-10数据集上具有和不具有批次归一化层的模型上,分别具有8位、6位和4位精度。我们还介绍了后训练量化方法的两种未来变体,即混合量化和非均匀量化

英文摘要:

Existing quantization aware training methods attempt to compensate for the quantization loss by leveraging on training data, like most of the post-training quantization methods, and are also time consuming. Both these methods are not effective for privacy constraint applications as they are tightly coupled with training data. In contrast, this paper proposes a data-independent post-training quantization scheme that eliminates the need for training data. This is achieved by generating a faux dataset, hereafter referred to as Retro-Synthesis Data, from the FP32 model layer statistics and further using it for quantization. This approach outperformed state-of-the-art methods including, but not limited to, ZeroQ and DFQ on models with and without Batch-Normalization layers for 8, 6, and 4 bit precisions on ImageNet and CIFAR-10 datasets. We also introduced two futuristic variants of post-training quantization methods namely Hybrid Quantization and Non-Uniform Quantization

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑