面向 DETR 检测器的 Fisher 感知量化与关键类别目标
Fisher-aware Quantization for DETR Detectors with Critical-category Objectives
AI总结:
本文针对量化对 DETR 检测器中关键类别性能的细粒度影响,提出基于 Fisher 信息的混合精度量化与 Fisher-trace 正则化方法,有效缓解关键类别过拟合并显著提升量化模型的关键类别 mAP。
摘要:
量化对深度学习模型整体性能的影响是一个已被深入研究的问题。然而,在更细粒度层面上理解和缓解其影响仍然不足,尤其是对于同时具有分类和回归目标的物体检测等更难的任务。本文将任务关键类别子集的性能,即关键类别性能,定义为检测任务中一个至关重要但在很大程度上被忽视的细粒度目标。我们在类别粒度上分析了量化的影响,并提出了改进关键类别性能的方法。具体而言,我们发现某些关键类别对量化具有更高的敏感性,并且在量化感知训练(QAT)后容易出现过拟合。为解释这一现象,我们利用 Fisher 信息框架,在理论和实证上建立了这些类别的性能差距与相应损失景观之间的联系。基于这一证据,我们应用了一种 Fisher 感知的混合精度量化方案,并在关键类别损失景观上对 QAT 施加 Fisher-trace 正则化。所提出的方法提升了基于 Transformer 的量化 DETR 检测器的关键类别指标。在模型更大、类别数量更多、过拟合更严重的情况下,这些改进更加显著。例如,在 COCO Panoptic 数据集中受影响最严重的关键类别上,我们的方法使 4-bit DETR-R50 和 Deformable DETR 分别获得了 10.4% 和 14.5% 的 mAP 提升。
英文摘要:
The impact of quantization on the overall performance of deep learning models is a well-studied problem. However, understanding and mitigating its effects on a more fine-grained level is still lacking, especially for harder tasks such as object detection with both classification and regression objectives. This work defines the performance for a subset of task-critical categories, i.e. the critical-category performance, as a crucial yet largely overlooked fine-grained objective for detection tasks. We analyze the impact of quantization at the category-level granularity, and propose methods to improve performance for the critical categories. Specifically, we find that certain critical categories have a higher sensitivity to quantization, and are prone to overfitting after quantization-aware training (QAT). To explain this, we provide theoretical and empirical links between their performance gaps and the corresponding loss landscapes with the Fisher information framework. Using this evidence, we apply a Fisher-aware mixed-precision quantization scheme, and a Fisher-trace regularization for the QAT on the critical-category loss landscape. The proposed methods improve critical-category metrics of the quantized transformer-based DETR detectors. They are even more significant in case of larger models and higher number of classes where the overfitting becomes more severe. For example, our methods lead to 10.4% and 14.5% mAP gains for, correspondingly, 4-bit DETR-R50 and Deformable DETR on the most impacted critical classes in the COCO Panoptic dataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。