The substantial computational costs of diffusion models, especially due to the repeated denoising steps necessary for high-quality image generation, present a major obstacle to their widespread adoption. While several studies have attempted to address this issue by reducing the number of score function evaluations (NFE) using advanced ODE solvers without fine-tuning, the decreased number of denoising iterations misses the opportunity to update fine details, resulting in noticeable quality degradation. In our work, we introduce an advanced acceleration technique that leverages the temporal redundancy inherent in diffusion models. Reusing feature maps with high temporal similarity opens up a new opportunity to save computation resources without compromising output quality. To realize the practical benefits of this intuition, we conduct an extensive analysis and propose a novel method, FRDiff. FRDiff is designed to harness the advantages of both reduced NFE and feature reuse, achieving a Pareto frontier that balances fidelity and latency trade-offs in various generative tasks.
Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformers
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China(人工智能学院,北京师范大学,北京);Institute of Artificial Intelligence(人工智能研究所);Future Networks, Beijing Normal University, Zhuhai 519087, China(未来网络,北京师范大学,珠海)
BWCache: Accelerating Video Diffusion Transformers through Block-Wise Caching
School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China(人工智能学院,北京师范大学,北京);Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, China(人工智能与未来网络研究院,北京师范大学,珠海)
SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation
The University of Hong Kong(香港大学);Huawei Noah’s Ark Lab(华为诺亚实验室);Peking University(北京大学);xAI;Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学)
MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference
Data Science & Artificial Intelligence Research Institute, China Unicom(中国unicom数据科学与人工智能研究 institute);Unicom Data Intelligence, China Unicom(中国unicom数据智能);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)
Not All Frames Deserve Full Computation: Accelerating Autoregressive Video Generation via Selective Computation and Predictive Extrapolation
Institute of Artificial Intelligence and Future Networks, Beijing Normal University(北京师范大学人工智能与未来网络研究院);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);Shenzhen University of Advanced Technology(深圳理工大学)
Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free
Shanghai Jiao Tong University(上海交通大学);China University of Mining and Technology(中国矿业大学);University of Electronic Science and Technology of China(电子科技大学);Tsinghua University(清华大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraints
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);SKL-IOTSC, CIS, University of Macau(澳门大学SKL-IOTSC、CIS);Huazhong University of Science and Technology(华中科技大学);The Chinese University of Hong Kong(香港中文大学)
Token Caching for Diffusion Transformer Acceleration
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所);School of Information and Electronic Engineering, Shandong Technology and Business University(山东科技职业大学信息与电子工程学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);PeopleAl, Inc.(PeopleAl公司);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Terminal Technology Department, Alipay, Ant Group(蚂蚁集团支付宝终端技术部)
RAPID^3: Tri-Level Reinforced Acceleration Policies for Diffusion Transformer
National University of Singapore(新加坡国立大学);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Hupan Lab(鸿篇实验室);ZIP Lab, Zhejiang University(浙江大学ZIP实验室);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
Sparse ActionGen: Accelerating Diffusion Policy with Real-time Pruning
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Department of Computer Science(计算机科学系)
BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration
Hefei University of Technology(合肥工业大学);University of Science and Technology of China(中国科学技术大学);Anhui University of Science and Technology(安徽理工大学);University of Macau(澳门大学)