Token Caching for Diffusion Transformer Acceleration
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所);School of Information and Electronic Engineering, Shandong Technology and Business University(山东科技职业大学信息与电子工程学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);PeopleAl, Inc.(PeopleAl公司);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Terminal Technology Department, Alipay, Ant Group(蚂蚁集团支付宝终端技术部)
Diffusion transformers have gained substantial interest in diffusion generative modeling due to their outstanding performance. However, their computational demands, particularly the quadratic complexity of attention mechanisms and multi-step inference processes, present substantial bottlenecks that limit their practical applications. To address these challenges, we propose TokenCache, a novel acceleration method that leverages the token-based multi-block architecture of transformers to reduce redundant computations. TokenCache tackles three critical questions: (1) Which tokens should be pruned and reused by the caching mechanism to eliminate redundancy? (2) Which blocks should be targeted for efficient caching? (3) At which time steps should caching be applied to balance speed and quality? In response to these challenges, TokenCache introduces a Cache Predictor that hierarchically addresses these issues by (1) Token pruning: assigning importance scores to each token to determine which tokens to prune and reuse; (2) Block selection: allocating pruning ratio to each block to adaptively select blocks for caching; (3) Temporal Scheduling: deciding at which time steps to apply caching strategies. Experimental results across various models demonstrate that TokenCache achieves an effective trade-off between generation quality and inference speed for diffusion transformers.
Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free
Shanghai Jiao Tong University(上海交通大学);China University of Mining and Technology(中国矿业大学);University of Electronic Science and Technology of China(电子科技大学);Tsinghua University(清华大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers
School of Computer Science, Peking University(北京大学计算机科学学院);School of Software Engineering, University of Electronic Science and Technology of China(电子科技大学软件工程学院)
Shanghai Jiao Tong University(上海交通大学);South China University of Technology(南方科技大学);Beihang University(北航);Huawei Technologies Ltd(华为技术有限公司);Shanghai AI Lab(上海人工智能实验室);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Accelerating Diffusion Transformers with Token-wise Feature Caching
Shanghai Jiao Tong University(上海交通大学);University of Electronic Science & Technology of China(电子科技大学);Sichuan University(四川大学);National University of Defense Technology(国防科技大学);Zhejiang University(浙江大学)
针对 Diffusion Transformer 计算成本高的问题,提出逐 token 自适应特征缓存,并按层类型与深度设置缓存比例,在 PixArt-α、OpenSora 和 DiT 上免训练实现近无损加速。
VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching
School of Computer Science, University of Sydney(悉尼大学计算机科学学院);John Hopcropt Center for Computer Science, Shanghai Jiao Tong University(上海交通大学约翰·霍普克罗夫特计算机科学中心)
Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
School of Computer Science and Technology, University of Science and Technology of China(计算机科学与技术学院,中国科学技术大学);Suzhou Institute of Advanced Research, University of Science and Technology of China(苏州先进研究院,中国科学技术大学)
VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness
School of Computer Science, Peking University(北京大学计算机科学系);School of Computer Science, China University of Geosciences (Wuhan)(中国地质大学(武汉)计算机科学系);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学系)
FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
Yale University(耶鲁大学);Columbia University(哥伦比亚大学);University of California, Los Angeles(加州大学洛杉矶分校);University of Wisconsin--Madison(威斯康星大学麦迪逊分校);Michigan State University(密歇根州立大学)
MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference
Data Science & Artificial Intelligence Research Institute, China Unicom(中国unicom数据科学与人工智能研究 institute);Unicom Data Intelligence, China Unicom(中国unicom数据智能);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)
KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
Technical University of Munich(慕尼黑技术大学);Technical University of Darmstadt(达姆施塔特技术大学);Zhejiang University(浙江大学);Technische Universität Ilmenau(伊尔梅瑙技术大学)
SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching
Shanghai Jiao Tong University(上海交通大学);Shandong University(山东大学);University of Electronic Science and Technology of China(电子科学与技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Tsinghua University(清华大学);National University of Singapore(新加坡国立大学)
Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models
Shanghai Jiao Tong University(上海交通大学);University of Electronic Science and Technology of China(电子科技大学);Shandong University(山东大学);Xiamen University(厦门大学);Fudan University(复旦大学)
OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models
Data Science & Artificial Intelligence Research Institute, China Unicom(中国联通数据科学与人工智能研究院);Unicom Data Intelligence, China Unicom(中国联通联通数据智能);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(教育部多媒体可信感知与高效计算重点实验室,厦门大学);Shandong Inspur Database Technology Co.,Ltd(山东浪潮数据库技术有限公司)
Recency/Frequency Adaptive KV Caching for Large Language Model Serving
Department of Computer Science, Johns Hopkins University, Baltimore, USA(约翰霍普金斯大学计算机科学系);Argonne National Laboratory, Lemont, USA(阿贡国家实验室);Parasail, Inc., San Mateo, USA(Parasail公司)