Deploying neural networks on the edge has become increasingly important as deep learning is being applied in an increasing amount of applications. At the edge computing hardware typically has limited resources disallowing to run neural networks with high complexity. To reduce the complexity of neural networks a wide range of quantization methods have been proposed in recent years. This work proposes Constraint Guided Model Quantization (CGMQ), which is a quantization aware training algorithm that uses an upper bound on the computational resources and reduces the bit-widths of the parameters of the neural network. CGMQ does not require the tuning of a hyperparameter to result in a mixed precision neural network that satisfies the predefined computational cost constraint, while prior work does. It is shown on MNIST and CIFAR10 that the performance of CGMQ is competitive with state-of-the-art quantization aware training algorithms, while guaranteeing the satisfaction of an upper bound on the computational complexity defined by the computational resources of the on edge hardware.
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance
Department of XXX, University of YYY, Location, Country；School of ZZZ, Institute of WWW, Location, Country；Department of Computer Science；Engineering, Seoul National University；Neural Processing Research Center；Samsung AI Lab, Montreal；Google
HESTIA: A Hessian-Guided Differentiable Quantization-Aware Training Framework for Extremely Low-Bit LLMs
School of Software and Microelectronics, Peking University, Beijing, China（软件与微电子学院，北京大学，北京，中国）；School of Computer Science, Peking University, Beijing, China（计算机科学学院，北京大学，北京，中国）
CALM: A CKA-Guided Adaptive Layer-Wise Modularization Framework for LLM Quantization
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Zhejiang University（浙江大学）；University of Science and Technology of China（中国科学技术大学）
Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization
Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hebei Province Key Laboratory of Big Data Science and Intelligent Technology（河北省大数据科学与智能技术重点实验室）
Multidisciplinary Faculty of Nador, Mohammed Premier University, Oujda 60000, Morocco（摩洛哥奥伊达大学多学科学院）；Laboratory LaSTI, ENSAK, Sultan Moulay Slimane University, Khouribga 54000, Morocco（摩洛哥库尔希布加大学LaSTI实验室）
FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices
School of Computing and Information Technology, Great Bay University（计算与信息学院，大亚湾大学）；Beijing Institute of Technology（北京理工大学）；University of Warwick（沃里克大学）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））；Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）
Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
Peng Cheng Laboratory（鹏城实验室）；The University of Tokyo（东京大学）；Southern University of Science and Technology（南方科技大学）；Peking University（北京大学）；Malardalen University（马拉达伦大学）；LocationMind Inc.（LocationMind公司）
Optimizing Learned Image Compression on Scalar and Entropy-Constraint Quantization
Fraunhofer Institute for Telecommunications – Heinrich Hertz Institute（弗劳恩霍夫电信研究所——海因里希·赫兹研究所）；Institute of Computer Science（计算机科学研究所）；Free University of Berlin（柏林自由大学）；Department of Telecommunication Systems（电信系统系）
