NIPQ：基于噪声代理的集成伪量化
NIPQ: Noise proxy-based Integrated Pseudo-Quantization
AI总结：
本文提出NIPQ，一种基于噪声代理的集成伪量化方法，通过在伪量化框架上集成截断思想，统一支持激活和权重的伪量化，避免了STE的不稳定性，并在视觉和语言应用中大幅优于现有量化算法。
摘要：
直通估计器（STE）通过近似使梯度流能够跨越不可微函数，在量化感知训练（QAT）相关研究中备受青睐。然而，STE在QAT期间会导致收敛不稳定，在低精度下引起显著的质量下降。近期，伪量化训练被提出作为一种替代方法，使用伪量化噪声代替STE来更新可学习参数。在本研究中，我们提出了一种新颖的基于噪声代理的集成伪量化（NIPQ），通过在伪量化框架上集成截断思想，实现对激活和权重的伪量化统一支持。NIPQ通过梯度下降更新所有量化参数（如位宽和截断边界）以及网络参数，而不会出现STE的不稳定性。根据我们广泛的实验，NIPQ在各种视觉和语言应用中大幅优于现有的量化算法。
英文摘要：
Straight-through estimator (STE), which enables the gradient flow over the non-differentiable function via approximation, has been favored in studies related to quantization-aware training (QAT). However, STE incurs unstable convergence during QAT, resulting in notable quality degradation in low precision. Recently, pseudoquantization training has been proposed as an alternative approach to updating the learnable parameters using the pseudo-quantization noise instead of STE. In this study, we propose a novel noise proxy-based integrated pseudoquantization (NIPQ) that enables unified support of pseudoquantization for both activation and weight by integrating the idea of truncation on the pseudo-quantization framework. NIPQ updates all of the quantization parameters (e.g., bit-width and truncation boundary) as well as the network parameters via gradient descent without STE instability. According to our extensive experiments, NIPQ outperforms existing quantization algorithms in various vision and language applications by a large margin.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
