扩散 Transformer(DiTs)将 Transformer 架构与扩散模型相结合。然而,其计算复杂度对实时应用和 AI 系统的可持续性造成了显著限制。在本研究中,我们旨在通过模型量化来提升计算效率,即以较低精度表示权重和激活值。我们引入了多区域量化(MRQ),通过为子区域分配两个缩放参数来解决 DiT 块中网络值的不对称分布问题。此外,我们提出了时间分组量化(TGQ),以减少由激活值的时间变化引起的量化误差。实验结果表明,所提算法在 W8A8 精度下仅以 FID 增加 0.29 的代价,就实现了与原始全精度模型相当的性能。此外,在 W6A6 精度下,该方法优于其他基线方法,从而证实了其适用于低位量化的能力。这些结果凸显了我们的方法在实现高效实时生成模型方面的潜力。
英文摘要:
Diffusion transformers (DiTs) combine transformer architectures with diffusion models. However, their computational complexity imposes significant limitations on real-time applications and sustainability of AI systems. In this study, we aim to enhance the computational efficiency through model quantization, which represents the weights and activation values with lower precision. Multi-region quantization (MRQ) is introduced to address the asymmetric distribution of network values in DiT blocks by allocating two scaling parameters to sub-regions. Additionally, time-grouping quantization (TGQ) is proposed to reduce quantization error caused by temporal variation in activations. The experimental results show that the proposed algorithm achieves performance comparable to the original full-precision model with only a 0.29 increase in FID at W8A8. Furthermore, it outperforms other baselines at W6A6, thereby confirming its suitability for low-bit quantization. These results highlight the potential of our method to enable efficient real-time generative models.
Sampling 3D Molecular Conformers with Diffusion Transformers
Technical University Berlin(技术大学柏林);BIFOLD Berlin(柏林BIFOLD);Google DeepMind(谷歌DeepMind);MPI for Informatics, Saarbrücken(萨尔布吕肯信息研究所);Department of Artificial Intelligence, Korea University(韩国大学人工智能系)
MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers
School of Computer Science, Peking University(北京大学计算机科学学院);School of Software Engineering, University of Electronic Science and Technology of China(电子科技大学软件工程学院)
A-SelecT: Automatic Timestep Selection for Diffusion Transformer Representation Learning
University of Missouri–Kansas City(密苏里大学堪萨斯城分校);U. S. Naval Research Laboratory(美国海军研究实验室);Lamar University(拉马尔大学);University of Florida(佛罗里达大学);University of Texas Rio Grande Valley(德克萨斯大学里奥格兰德河谷分校);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校);Meta AI;Rochester Institute of Technology(罗切斯特理工学院)
ChebBooster: A Training-Free Approach for Efficient Diffusion Transformer Inference via Chebyshev-Inspired Extrapolation
College of Electronics and Information Engineering, Shenzhen University(深圳大学电子与信息工程学院);School of Artificial Intelligence, Shenzhen University(深圳大学人工智能学院)
Diffusion Transformers are Provably Optimal In-context Generators
National University of Singapore(新加坡国立大学);RIKEN AIP(日本理化学研究所先进智能研究中心);The University of Tokyo(东京大学);A*STAR CFAR(新加坡科技研究局计算与人工智能研究中心);Nanyang Technological University(南洋理工大学)
Shanghai Jiao Tong University(上海交通大学);South China University of Technology(南方科技大学);Beihang University(北航);Huawei Technologies Ltd(华为技术有限公司);Shanghai AI Lab(上海人工智能实验室);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Scaling Diffusion Transformers Efficiently via $μ$P
Gaoling School of AI, Renmin University of China(中国人民大学人工智能学院);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Engineering Research Center of Next-Generation Intelligent Search(下一代智能搜索与推荐工程研究中心);Tsinghua University(清华大学)