AdaTSQ: 通过时间敏感量化推动扩散变换器的帕累托前沿
AdaTSQ: Pushing the Pareto Frontier of Diffusion Transformers via Temporal-Sensitivity Quantization
AI总结：
AdaTSQ通过时间敏感量化方法提升扩散变换器的效率与质量，优于现有方法。
摘要：
扩散变换器（DiTs）已涌现出作为高保真图像和视频生成的最先进骨干网络。然而，其庞大的计算成本和内存占用阻碍了在边缘设备上的部署。尽管后训练量化（PTQ）已被证明在大型语言模型（LLMs）中有效，但直接应用现有方法到DiTs会产生次优结果，因为忽略了扩散过程中固有的时间动态特性。在本文中，我们提出AdaTSQ，一种新的PTQ框架，通过利用DiTs的时间敏感性来推动效率和质量的帕累托前沿。首先，我们提出了一种帕累托感知的时间步动态位宽分配策略。我们将量化策略搜索建模为一个约束路径寻找问题。我们利用由端到端重建误差引导的束搜索算法，动态分配不同时间步的层位宽。其次，我们提出了一种Fisher引导的时间校准机制。它利用时间Fisher信息来优先校准高敏感时间步的数据，无缝整合到基于Hessian的权重优化中。在四个先进的DiTs（例如Flux-Dev、Flux-Schnell、Z-Image和Wan2.1）上的广泛实验表明，AdaTSQ显著优于SVDQuant和ViDiT-Q等最先进的方法。我们的代码将在https://github.com/Qiushao-E/AdaTSQ上发布。
英文摘要：
Diffusion Transformers (DiTs) have emerged as the state-of-the-art backbone for high-fidelity image and video generation. However, their massive computational cost and memory footprint hinder deployment on edge devices. While post-training quantization (PTQ) has proven effective for large language models (LLMs), directly applying existing methods to DiTs yields suboptimal results due to the neglect of the unique temporal dynamics inherent in diffusion processes. In this paper, we propose AdaTSQ, a novel PTQ framework that pushes the Pareto frontier of efficiency and quality by exploiting the temporal sensitivity of DiTs. First, we propose a Pareto-aware timestep-dynamic bit-width allocation strategy. We model the quantization policy search as a constrained pathfinding problem. We utilize a beam search algorithm guided by end-to-end reconstruction error to dynamically assign layer-wise bit-widths across different timesteps. Second, we propose a Fisher-guided temporal calibration mechanism. It leverages temporal Fisher information to prioritize calibration data from highly sensitive timesteps, seamlessly integrating with Hessian-based weight optimization. Extensive experiments on four advanced DiTs (e.g., Flux-Dev, Flux-Schnell, Z-Image, and Wan2.1) demonstrate that AdaTSQ significantly outperforms state-of-the-art methods like SVDQuant and ViDiT-Q. Our code will be released at https://github.com/Qiushao-E/AdaTSQ.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
