Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization
Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hebei Province Key Laboratory of Big Data Science and Intelligent Technology（河北省大数据科学与智能技术重点实验室）
Xi Chen, Ming Li, Junxi Li, Changsheng Li, Peisong Wang, Lizhong Ding, Ye Yuan, Guoren Wang
Weight-only post-training quantization (PTQ) is crucial for efficient Large Language Model (LLM) deployment but suffers from accuracy degradation caused by weight and activation outliers. Existing mitigation strategies often face critical limitations: they either yield insufficient outlier suppression or incur significant deployment inefficiencies, such as inference latency, heavy preprocessing, or reliance on complex operator fusion. To resolve these limitations, we leverage a key insight: over-parameterized LLMs often converge to Flat Minima, implying a vast equivalent solution space where weights can be adjusted without compromising accuracy. Building on this, we propose Astro, an Activation-guided Structured Regularization framework designed to suppress the negative effects of outliers in a hardware-friendly and efficient manner. Leveraging the activation-guided regularization objective, Astro actively reconstructs intrinsically robust weights, aggressively suppressing weight outliers corresponding to high-magnitude activations without sacrificing model accuracy. Crucially, Astro introduces zero inference latency and is orthogonal to mainstream quantization methods like GPTQ. Extensive experiments show that Astro achieves highly competitive performance; notably, on LLaMA-2-7B, it achieves better performance than complex learning-based rotation methods with almost 1/3 of the quantization time.
OffQ: Taming Structured Outliers in LLM Quantization by Offsetting
School of Computer and Communication Sciences, EPFL, Switzerland（瑞士联邦理工学院计算机与通信科学学院）；Huawei, Switzerland（华为公司）；Swiss Data Science Center, ETHZ & EPFL, Switzerland（瑞士数据科学中心，苏黎世联邦理工学院与联邦理工学院）
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（认知与决策智能复杂系统重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；AMD；Ritzz-AI
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University（应用信息与应用数学博士学院，奥布达大学）；John von Neumann Faculty of Informatics, Obuda University（约翰·冯·诺伊曼信息学院，奥布达大学）
InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization
School of Software Technology, Zhejiang University（浙江大学软件学院）；Ant Group（蚂蚁集团）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；China Mobile (Zhejiang) Research & Innovation Institute（中国移动（浙江）研究院）；Alibaba Cloud Computing（阿里云计算）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Wuhan AI Research（武汉人工智能研究院）；Objecteye Inc.（智视科技）
Pushing the Limits of Block Rotations in Post-Training Quantization
Advanced Micro Devices, Inc. (AMD)（Advanced Micro Devices公司）；State University of New York at Buffalo（纽约州立大学布法罗分校）；Norwegian University of Science（挪威科学与技术大学）
SmoothRot: Combining Channel-Wise Scaling and Rotation for Quantization-Friendly LLMs
Doctoral School of Applied Informatics and Applied Mathematics, Obuda University（应用信息学与应用数学博士学院，奥布达大学）；John von Neumann Faculty of Informatics, Obuda University（约翰·冯·诺依曼信息学院，奥布达大学）
TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control
Dept. of Comp. Sci. and Tech., Institute for AI, BNRist Center, THBI Lab, Tsinghua-Bosch Joint ML Center, Tsinghua University（计算机科学与技术系，人工智能研究所，BNRist中心，THBI实验室，清华-博世联合机器学习中心，清华大学）；Zhili College, Tsinghua University（紫荆学院，清华大学）；Bosch AI Research, Renningen, Germany（博世人工智能研究，德国Renningen）
The Curse and Blessing of Mean Bias in FP4-Quantized LLM Training
Fudan University（复旦大学）；University of Bath（巴斯大学）；Shanghai Innovation Institute（上海创新研究院）；University of Oxford（牛津大学）；Oxford Suzhou Centre for Advanced Research（牛津苏浙研究中心）；University of Colorado Boulder（科罗拉多大学波德格分校）；University of Michigan（密歇根大学）；Shenzhen Loop Area Institute（深圳河套学院）
CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts
Courant Institute of Mathematical Sciences, New York University（纽约大学库朗数学科学研究所）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Cerebras Systems Inc.（Cerebras系统公司）
Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition
Department of Computer Science, Yonsei University（yonsei大学计算机科学系）；Department of Artificial Intelligence, Yonsei University（yonsei大学人工智能系）；Department of Electronic & Electrical Convergence Engineering, Hongik University（hongik大学电子与电气融合工程系）
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
ZIP Lab, Monash University（莫纳什大学ZIP实验室）；SenseTime Research（商汤科技研究院）；Beihang University（北京航空航天大学）；School of Computer and Communication Sciences, EPFL（洛桑联邦理工学院计算机与通信科学学院）；University of Science and Technology Beijing（北京科技大学）
Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
Massachusetts Institute of Technology（麻省理工学院）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；IBM Research（IBM研究院）；IST Austria（奥地利科学技术研究所）；Neural Magic, Inc.（Neural Magic公司）
