Post-Training Quantization (PTQ) is crucial for efficient model deployment, yet its effectiveness on Ascend NPU remains under-explored compared to GPU architectures. This paper presents a case study of representative PTQ baselines applied to reasoning-oriented models such as DeepSeek-R1-Distill-Qwen series (1.5B/7B/14B) and QwQ-32B. We evaluate four distinct algorithms, including AWQ, GPTQ, SmoothQuant, and FlatQuant, to cover the spectrum from weight-only compression to advanced rotation-based methods. Our empirical results reveal significant platform sensitivity. While 4-bit weight-only quantization proves viable for larger models, aggressive 4-bit weight-activation schemes suffer from layer-wise calibration instability on the NPU, leading to logic collapse in long-context reasoning tasks. Conversely, standard 8-bit quantization remains numerically stable. Furthermore, a real-world INT8 deployment demonstrates that although optimized kernels reduce latency, dynamic quantization overheads currently limit end-to-end acceleration. These findings offer a practical reference for the feasibility and limitations of deploying quantized reasoning models on Ascend NPU.
An Empirical Study of openPangu Quantization on Ascend NPUs
State Key Laboratory of Complex & Critical Software Environment, Beihang University(北京航空航天大学复杂关键软件环境国家重点实验室);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院)
AscendOptimizer: Episodic Agent for Ascend NPU Operator Optimization
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Shanghai University of International Business and Economics(上海国际商务经济大学);Key Lab of Mathematics and Engineering Applications (MoE), East China Normal University(东华大学数学与工程应用重点实验室);School of Mathematical Sciences, East China Normal University(东华大学数学科学学院);Shenzhen Loop Area Institute (SLAI)(深圳环宇院)
ENEC: A Lossless AI Model Compression Method Enabling Fast Inference on Ascend NPUs
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Huawei Technologies Co., Ltd.(华为技术有限公司)
Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
Huawei(华为);Tsinghua University(清华大学);Shanghai AI Lab(上海人工智能实验室)