EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Diffusion models have achieved great success in image generation tasks. However, the lengthy denoising process and complex neural networks hinder their low-latency applications in real-world scenarios. Quantization can effectively reduce model complexity, and post-training quantization (PTQ), which does not require fine-tuning, is highly promising for compressing and accelerating diffusion models. Unfortunately, we find that due to the highly dynamic activations, existing PTQ methods suffer from distribution mismatch issues at both calibration sample level and reconstruction output level, which makes the performance far from satisfactory. In this paper, we propose EDA-DM, a standardized PTQ method that efficiently addresses the above issues. Specifically, at the calibration sample level, we extract information from the density and diversity of latent space feature maps, which guides the selection of calibration samples to align with the overall sample distribution; and at the reconstruction output level, we theoretically analyze the reasons for previous reconstruction failures and, based on this insight, optimize block reconstruction using the Hessian loss of layers, aligning the outputs of quantized model and full-precision model at different network granularity. Extensive experiments demonstrate that EDA-DM significantly outperforms the existing PTQ methods across various models and datasets. Our method achieves a 1.83 times speedup and 4 times compression for the popular Stable-Diffusion on MS-COCO, with only a 0.05 loss in CLIP score. Code is available at http://github.com/BienLuky/EDA-DM .
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；Ningbo Innovation Center, Zhejiang University（浙大宁波创新中心）；State Key Laboratory of Blockchain and Security, Zhejiang University（浙江大学区块链与安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
Tsinghua University（清华大学）；University of California Santa Barbara（加利福尼亚大学圣巴巴拉分校）；Microsoft（微软公司）；Infinigence AI（智源人工智能）；Shanghai Jiaotong University（上海交通大学）
LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization
Zhejiang University（浙江大学）；Tsinghua University（清华大学）；Westlake University（西湖大学）；The Hong Kong University of Science and Technology（香港科技大学）；OPPO AI Center（OPPO人工智能中心）；CAIR, HKISI-CAS（中国科学院香港中文大学深圳先进集成技术研究所）；MAIS CASIA（中国科学院自动化研究所多模态人工智能系统实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling
Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan（台湾大学电子工程研究所）；Institute of Information Science, Academia Sinica, Taiwan（中科院资讯研究所）；Graduate Institute of Communication Engineering, National Taiwan University, Taiwan（台湾大学通讯工程研究所）；Institute of Information Science and the Research Center for Information Technology Innovation, Academia Sinica, Taiwan（中科院资讯研究所及资讯科技创新研究中心）
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；MAC Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University（厦门大学信息学院人工智能系）；Alibaba（阿里巴巴）
Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization
Hefei University of Technology（合肥工业大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；National University of Singapore（新加坡国立大学）
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Department of Information Technology（信息科技系）；Electrical Engineering, ETH Zurich, Zurich, Switzerland（电气工程，苏黎世联邦理工学院，苏黎世，瑞士）；Qualcomm AI Research, Amsterdam, the Netherlands（高通人工智能研究，阿姆斯特丹，荷兰）；Department of Electrical, Electronic and Information Engineering（电气、电子与信息工程系）；University of Bologna, Bologna, Italy（博洛尼亚大学，博洛尼亚，意大利）；School of Electrical and Electronic Engineering（电气与电子工程学院）
HIH: Towards More Accurate Face Alignment via Heatmap in Heatmap
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Baidu Inc.（百度公司）
基于热图的回归克服了直接坐标回归缺乏空间和上下文信息的问题，并彻底改变了人脸对齐的任务。然而，它遭受的量化误差所造成的忽略亚像素坐标在图像去噪和网络降采样。在本文中，我们首先定量分析了量化误差的基准，占1/3以上的整体预测误差的最先进的方法。为了解决这个问题，我们提出了一种新的Heatmap In Heatmap（HIH）表示和一种坐标软分类（CSC）方法，它们无缝集成到经典的沙漏网络中。HIH表示使用嵌套的热图来共同表示坐标标签：一个称为整数热图的热图表示整数坐标，另一个称为十进制热图的热图表示子像素坐标。十进制热图的范围构成相应整数热图中的一个像素。此外，我们将偏移回归问题转化为一个区间分类问题，CSC将像素点的置信度视为区间的概率。同时，应用分布损失的CSC利用从高斯分布函数生成的软标签来指导偏移热图训练，这使得更容易学习坐标偏移的分布。在具有挑战性的基准数据集上进行的大量实验表明，我们的HIH可以实现最先进的结果。特别是，我们的HIH在WFLW上达到4.08 NME（归一化平均误差），在COFW上达到3.21，这大大超过了以前的方法。
GLF-Q: Global-Local Feature-based Quantization for Vision Transformers
State Key Laboratory of Novel Software Technology, Nanjing University（南京大学计算机软件新技术全国重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；Zhongguancun Academy（中关村学院）
MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models
institutetext: MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models Yue Wu Changyuan Wang Zixuan Wang Shilin Ma Yansong Tang（机构文本：MorphoQuant：多模态大语言模型的模态感知量化 Yue Wu 王昌元 王梓轩 马世林 唐彦松）
MAG: Multi-Modal Aligned Autoregressive Co-Speech Gesture Generation without Vector Quantization
Communication University of China（中国传媒大学）；China Mobile Research Institute（中国移动研究院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；The Chinese University of Hong Kong（香港中文大学）
