Quantization Error Reconstruction (QER) reduces accuracy loss in Post-Training Quantization (PTQ) by approximating weights as $\mathbf{W} \approx \mathbf{Q} + \mathbf{L}\mathbf{R}$, using a rank-$r$ correction to reconstruct quantization error. Prior methods devote the full rank budget to error reconstruction, which is suboptimal when $\mathbf{W}$ has intrinsic low-rank structure and quantization corrupts dominant directions. We propose Structured Residual Reconstruction (SRR), a rank-allocation framework that preserves the top-$k$ singular subspace of the activation-scaled weight before quantization, quantizes only the residual, and uses the remaining rank $r-k$ for error reconstruction. We derive a theory-guided criterion for selecting $k$ by balancing quantization-exposed energy and unrecoverable error under rank constraints. We further show that resulting $\mathbf{Q} + \mathbf{L}\mathbf{R}$ parameterization naturally supports Quantized Parameter-Efficient Fine-Tuning (QPEFT), and stabilizes fine-tuning via gradient scaling along preserved directions. Experiments demonstrate consistent perplexity reductions across diverse models and quantization settings in PTQ, along with a 5.9 percentage-point average gain on GLUE under 2-bit QPEFT. The project page is available at https://ai-isl.github.io/srr.
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Zhongguancun Academy（中关村学院）；City University of Hong Kong（香港城市大学）；NJUST（南京理工大学）
Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
TICLab, International University of Rabat, Morocco（拉巴特国际大学TIC实验室，摩洛哥）；College of Computing, University Mohammed VI Polytechnic, Morocco（摩洛哥穆莱·伊斯梅尔Polytechnic大学计算机学院）；School of Electronic and Electrical Engineering, University of Leeds, UK（利兹大学电子与电气工程学院）
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning
Carnegie Mellon University（卡内基梅隆大学）；Columbia University（哥伦比亚大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Petuum Inc.（Petuum公司）；Massachusetts Institute of Technology（麻省理工学院）
A Replicate-and-Quantize Strategy for Plug-and-Play Load Balancing of Sparse Mixture-of-Experts LLMs
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Minnesota Twin Cities（明尼苏达大学双城分校）；North Carolina State University（北卡罗来纳州立大学）；Meta AI；Stevens Institute of Technology（史蒂文斯理工学院）
Derived Fields Increase the Recoverable Information in Budgeted Neural PDE Surrogate
School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；State Key Laboratory of Ocean Sensing & Ocean College, Zhejiang University（浙江大学海洋传感与海洋学院国家重点实验室）；Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology（麻省理工学院卡弗里天体物理与空间研究所）
China University of Geoscience Beijing（中国地质大学（北京））；Hong Kong University of Science and Technology（香港科学与技术大学）；Hong Kong Polytechnic University（香港理工大学）；Jilin University（吉林大学）；City University of Hong Kong（香港城市大学）；Chinese Academy of Sciences（中国科学院）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））
Auto-Configured Networks for Multi-Scale Multi-Output Time-Series Forecasting
National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University（东北大学工业智能与系统优化国家级前沿科学中心）；Innovative Artificial Intelligence Cluster, Digital Future Institute, De Montfort University（德蒙福特大学数字未来研究所创新人工智能集群）；Key Laboratory of Data Analytics and Optimization for Smart Industry (Northeastern University), Ministry of Education（教育部智能工业数据解析与优化重点实验室（东北大学））
