发表机构
Sony Interactive Entertainment（索尼互动娱乐）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出I-Parakeet，一种纯整数实现的Conformer ASR模型，通过整数注意力、Swish近似和激活校准，在移动NPU上实现4.97% WER，速度比CPU快7.5倍。
AI 中文摘要
本文提出I-Parakeet，这是NVIDIA Parakeet-CTC（0.6B参数）的纯整数实现，可在智能手机NPU上运行，无需任何浮点运算或CPU回退。现代Conformer ASR模型因其规模庞大而难以部署在边缘设备上，且量化模型在数值敏感操作上仍会回退到浮点运算，这阻碍了它们充分利用移动NPU等整数加速器。为实现此目标，我们做出了三项贡献。首先，我们推导了Conformer核心相对位置自注意力的整数公式，将两个具有不同量化尺度的得分分支和相对位移融合为纯整数操作。其次，我们引入了一种极小极大优化的Swish近似，最小化Swish输出的最大误差。第三，逐层激活范围分析产生了两项针对性补救措施：对BatchNorm输出采用INT16网格，对重尾预编码器激活采用百分位校准。I-Parakeet在LibriSpeech test-other上实现了4.97%的词错误率，在Qualcomm NPU上以0.048的实时因子运行，比CPU基线快7.5倍。
英文摘要
In this paper, we propose I-Parakeet, an integer-only implementation of NVIDIA's Parakeet-CTC (0.6B parameters) that runs on a smartphone NPU without any floating-point operator or CPU fallback. Modern Conformer ASR models are hard to deploy on edge devices because of their size, and quantized models still fall back to floating point for numerically sensitive operations. This prevents them from fully exploiting integer accelerators such as mobile NPUs. To achieve this, our contributions are threefold. First, we derive an integer formulation of the relative-positional self-attention at the core of the Conformer. We fuse its two score branches with different quantization scales and the relative shift into integer-only operations. Second, we introduce a minimax-optimized Swish approximation that minimizes the maximum error of the Swish output. Third, a layer-wise range analysis of activations yields two targeted remedies: an INT16 grid for the BatchNorm output and percentile calibration for the heavy-tailed pre-encoder activations. I-Parakeet achieves 4.97% WER on LibriSpeech test-other, running on a Qualcomm NPU at a real-time factor of 0.048, 7.5x faster than a CPU baseline.
CommentsUnder review
