为轻量级检测变压器实现完全仅整数推理

中文摘要 AI 辅助

视觉Transformer检测器如今已接近CNN的精度，但由于包括可变形注意力、特征融合和非线性激活函数等关键组件与整数运算不兼容，难以在NPUs和微控制器上部署。现有量化检测器要么保留如Softmax、GELU和LayerNorm等算子，要么专注于重量级主干，使得轻量级检测变压器缺乏端到端整数实现。本文提出I-LW-DETR，首个完全仅整数的轻量级DETR，其前向传播中的每个操作都用整数运算执行。它基于三个关键组件构建，实验结果表明该量化管道能在不同模型规模下持续生成高效的完全仅整数模型，在降低模型大小约3.6倍和计算成本超一个数量级的同时，精度仅适度下降。

英文摘要

Vision Transformer detectors now approach the accuracy of CNNs but remain difficult to deploy on NPUs and microcontrollers because key components, including deformable attention, feature fusion, and nonlinear activation functions, are not natively compatible with integer arithmetic. Existing quantized detectors either retain operators such as Softmax, GELU, and LayerNorm or focus on heavyweight backbones, leaving lightweight detection transformers without an end-to-end integer implementation. We address this gap with I-LW-DETR, the first fully integer-only lightweight DETR, in which every operation in the forward pass, including transformer nonlinearities, is executed in integer arithmetic. I-LW-DETR is built upon three key components: a scale-preserving split convolution that assigns independent activation scale to each branch of the multi-scale projector; SD-ShiftGELU, a sign-dependent GELU approximation that preserves element-wise behavior while avoiding the accuracy degradation; and a constrained Shiftmax that maintains stable Softmax normalization. Experimental results demonstrate that the proposed quantization pipeline consistently produces efficient fully integer-only models across different model scales. Across all model scales, the proposed pipeline incurs only a moderate accuracy degradation while reducing the model size by approximately $3.6\times$ and the computational cost by more than one order of magnitude.

发表机构

• CEA（法国国家科学研究中心）

机构由 AI 辅助整理，请以论文原文为准。