arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2306.04879cs.LG
收藏

利用层间依赖增强 Hessian 以实现混合精度训练后量化

Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization

University of Notre Dame(圣母大学);Google LLC(谷歌有限责任公司)

Clemens JS Schaefer, Navid Lambert-Shirzad, Xiaofan Zhang, Chiachen Chou, Tom Jablin, Jian Li, Elfie Guo, Caitlin Stanton, Siddharth Joshi, Yu Emma Wang

AI总结:

提出一种融合 Hessian 二阶信息与层间依赖的混合精度 PTQ 方法,通过二分搜索在可接受精度损失内优化精度-延迟权衡。

摘要:

由于模型复杂性和参数数量不断增加,以低延迟高效部署神经网络模型正变得更具挑战性。模型量化提供了一种同时减少内存占用和计算需求的解决方案。然而,激进量化可能导致不可接受的模型精度损失,因为模型中不同层对数值缺陷的敏感性存在差异。为应对这一挑战,我们提出一种混合精度训练后量化(PTQ)方法,根据网络中张量的具体需求为其分配不同的数值精度,从而在保持模型精度的同时减少内存占用并改善延迟。以往工作依赖逐层 Hessian 信息来确定数值精度,但正如我们所证明的,Hessian 估计通常不足以确定有效的层敏感性排序。我们通过利用额外信息增强估计的 Hessian 来捕捉层间依赖,从而解决该问题。我们证明,这会在多个模型上沿精度-延迟 Pareto 前沿持续提升 PTQ 性能。我们的方法结合二阶信息和层间依赖来引导二分搜索,在用户可配置的模型精度下降范围内寻找量化配置。我们在 ResNet50、MobileNetV2 和 BERT 模型上评估了方法的有效性。实验表明,与 16 位基线相比,延迟分别降低 25.48%、21.69% 和 33.28%,同时模型精度保持在基线模型的 99.99% 以内。

英文摘要:

Efficiently serving neural network models with low latency is becoming more challenging due to increasing model complexity and parameter count. Model quantization offers a solution which simultaneously reduces memory footprint and compute requirements. However, aggressive quantization may lead to an unacceptable loss in model accuracy owing to differences in sensitivity to numerical imperfection across different layers in the model. To address this challenge, we propose a mixed-precision post training quantization (PTQ) approach that assigns different numerical precisions to tensors in a network based on their specific needs, for a reduced memory footprint and improved latency while preserving model accuracy. Previous works rely on layer-wise Hessian information to determine numerical precision, but as we demonstrate, Hessian estimation is typically insufficient in determining an effective ordering of layer sensitivities. We address this by augmenting the estimated Hessian with additional information to capture inter-layer dependencies. We demonstrate that this consistently improves PTQ performance along the accuracy-latency Pareto frontier across multiple models. Our method combines second-order information and inter-layer dependencies to guide a bisection search, finding quantization configurations within a user-configurable model accuracy degradation range. We evaluate the effectiveness of our method on the ResNet50, MobileNetV2, and BERT models. Our experiments demonstrate latency reductions compared to a 16-bit baseline of $25.48\%$, $21.69\%$, and $33.28\%$ respectively, while maintaining model accuracy to within $99.99\%$ of the baseline model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑