DiTAS：通过增强激活平滑量化扩散Transformer
DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
AI总结：
针对扩散Transformer部署成本高的问题，本文提出无数据训练后量化方法DiTAS，通过时间聚合激活平滑、逐层网格搜索和免训练LoRA模块实现W4A8量化，性能接近全精度模型。
摘要：
扩散Transformer（DiTs）近期因其在视觉生成方面的增强能力而引起了工业界和学术界的广泛关注，其性能超越了采用U-Net的传统扩散模型。然而，DiTs性能的提升以更高的参数量和实现成本为代价，这极大地限制了它们在手机等资源受限设备上的部署。我们提出DiTAS，一种用于高效DiT推理的无数据训练后量化（PTQ）方法。DiTAS依靠所提出的时间聚合平滑技术来减轻输入激活中通道维度离群值的影响，从而在极低位宽下实现更低的量化误差。为了进一步提升量化后DiT的性能，我们采用逐层网格搜索策略来优化平滑因子。此外，我们集成了一个无需训练的LoRA模块用于权重量化，利用交替优化在无需额外微调的情况下最小化量化误差。实验结果表明，我们的方法能够实现DiTs的4位权重、8位激活（W4A8）量化，同时保持与全精度模型相当的性能。
英文摘要：
Diffusion Transformers (DiTs) have recently attracted significant interest from both industry and academia due to their enhanced capabilities in visual generation, surpassing the performance of traditional diffusion models that employ U-Net. However, the improved performance of DiTs comes at the expense of higher parameter counts and implementation costs, which significantly limits their deployment on resource-constrained devices like mobile phones. We propose DiTAS, a data-free post-training quantization (PTQ) method for efficient DiT inference. DiTAS relies on the proposed temporal-aggregated smoothing techniques to mitigate the impact of the channel-wise outliers within the input activations, leading to much lower quantization error under extremely low bitwidth. To further enhance the performance of the quantized DiT, we adopt the layer-wise grid search strategy to optimize the smoothing factor. Moreover, we integrate a training-free LoRA module for weight quantization, leveraging alternating optimization to minimize quantization errors without additional fine-tuning. Experimental results demonstrate that our approach enables 4-bit weight, 8-bit activation (W4A8) quantization for DiTs while maintaining comparable performance as the full-precision model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
