Large language models (LLMs) have achieved outstanding performance across a wide range of natural language processing tasks, but their enormous parameter counts impose ubstantial memory and computational overheads. This challenge is particularly critical in NPU-based on-device environments, where FP16/FP32 computation is inefficient and integer (INT) quantization is therefore essential. However, existing methods, including ZeroQuant, LLM.int8(), and SmoothQuant, do not fully address input-activation outliers and the associated hardware inefficiencies. To overcome these limitations, we propose MUXQ (Mixed-to-Uniform Quantization). MUXQ detects outlier channels in input activations and introduces a small auxiliary matrix that redistributes outlier magnitudes across channels, thereby alleviating the outlier problem. This enables even activation outliers to be quantized at low-precision INT levels while preserving a hardware-friendly computation structure. Experiments on GPT-2 models at three scales (0.1B, 0.3B, and 0.7B parameters) using the WikiText-2 dataset show that MUXQ consistently achieves lower perplexity than naive quantization. In particular, under per-tensor quantization, MUXQ quantizes both activations and weights to INT8 while maintaining accuracy close to that of FP16. With only modest computational overhead, MUXQ enables stable low-precision inference and can be readily combined with other quantization techniques. These results suggest that MUXQ provides a promising direction for efficient and accurate LLM inference on edge devices.
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；AMD；Ritzz-AI
Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization
Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hebei Province Key Laboratory of Big Data Science and Intelligent Technology（河北省大数据科学与智能技术重点实验室）
OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
School of Computer and Communication Sciences, EPFL, Switzerland（瑞士联邦理工学院计算机与通信科学学院）；Huawei, Switzerland（华为公司）；Swiss Data Science Center, ETHZ & EPFL, Switzerland（瑞士数据科学中心，苏黎世联邦理工学院与联邦理工学院）
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University（应用信息与应用数学博士学院，奥布达大学）；John von Neumann Faculty of Informatics, Obuda University（约翰·冯·诺伊曼信息学院，奥布达大学）
SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University（应用信息学与应用数学博士学院，奥布达大学）；John von Neumann Faculty of Informatics, Obuda University（约翰·冯·诺依曼信息学院，奥布达大学）
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
institutetext: MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models Yue Wu Changyuan Wang Zixuan Wang Shilin Ma Yansong Tang（机构文本：MorphoQuant：多模态大语言模型的模态感知量化 Yue Wu 王昌元 王梓轩 马世林 唐彦松）
Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
VCIP, College of Computer Science, Nankai University（南开大学计算机科学学院VCIP）；D-ITET, ETH Zürich（苏黎世联邦理工学院D-ITET）；OPPO Research Institute（OPPO研究院）；Department of Computing, Hong Kong Polytechnic University（香港理工大学计算机系）
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
School of Software Technology, Zhejiang University（浙江大学软件学院）；Ant Group（蚂蚁集团）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；Alibaba Cloud Computing（阿里云计算）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM
University of Arizona（亚利桑那大学）；Duke University（杜克大学）；Sungkyunkwan University（成均馆大学）；Panasonic AI Lab（松下人工智能实验室）；Korea Advanced Institute of Science and Technology（韩国科学技术院）
Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
Department of Computer Science, Yonsei University（yonsei大学计算机科学系）；Department of Artificial Intelligence, Yonsei University（yonsei大学人工智能系）；Department of Electronic & Electrical Convergence Engineering, Hongik University（hongik大学电子与电气融合工程系）
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
ZIP Lab, Monash University（莫纳什大学ZIP实验室）；SenseTime Research（商汤科技研究院）；Beihang University（北京航空航天大学）；School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）；University of Science and Technology Beijing（北京科技大学）
The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
Fudan University（复旦大学）；University of Bath（巴斯大学）；Shanghai Innovation Institute（上海创新研究院）；University of Oxford（牛津大学）；Oxford Suzhou Centre for Advanced Research（牛津苏浙研究中心）；University of Colorado Boulder（科罗拉多大学波德格分校）；University of Michigan（密歇根大学）；Shenzhen Loop Area Institute（深圳河套学院）
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Chinese University of Hong Kong（香港中文大学）
