Diffusion Transformers (DiT) have demonstrated remarkable generative capabilities but remain highly computationally expensive. Previous acceleration methods, such as pruning and distillation, typically rely on a fixed computational capacity, leading to insufficient acceleration and degraded generation quality. To address this limitation, we propose \textbf{Elastic Diffusion Transformer (E-DiT)}, an adaptive acceleration framework for DiT that effectively improves efficiency while maintaining generation quality. Specifically, we observe that the generative process of DiT exhibits substantial sparsity (i.e., some computations can be skipped with minimal impact on quality), and this sparsity varies significantly across samples. Motivated by this observation, E-DiT equips each DiT block with a lightweight router that dynamically identifies sample-dependent sparsity from the input latent. Each router adaptively determines whether the corresponding block can be skipped. If the block is not skipped, the router then predicts the optimal MLP width reduction ratio within the block. During inference, we further introduce a block-level feature caching mechanism that leverages router predictions to eliminate redundant computations in a training-free manner. Extensive experiments across 2D image (Qwen-Image and FLUX) and 3D asset (Hunyuan3D-3.0) demonstrate the effectiveness of E-DiT, achieving up to $\sim$2$\times$ speedup with negligible loss in generation quality. Code will be available at https://github.com/wangjiangshan0725/Elastic-DiT.
Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers
IDEAS NCBR;Jagiellonian University, Faculty of Mathematics and Computer Science(雅盖隆大学数学与计算机科学学院);Jagiellonian University, Doctoral School of Exact and Natural Sciences(雅盖隆大学精确与自然科学博士学院);University of Warsaw(华沙大学);Warsaw University of Technology(华沙技术大学);Tooploox
VolDiT: Controllable Volumetric Medical Image Synthesis with Diffusion Transformers
Institute for Artificial Intelligence in Cardiovascular Medicine, Medical Faculty of Heidelberg University, Heidelberg University(海德堡大学医学院心血管医学人工智能研究所);Department of Cardiology, Angiology, Pneumology, Heidelberg University Hospital(海德堡大学医院心脏病学、血管学、肺病学系);DZHK (German Centre for Cardiovascular Research), Partner Site Heidelberg/Mannheim(德国心血管研究中心(DZHK)海德堡/曼海姆合作站点);University Medical Center of the Johannes Gutenberg-University Mainz(美因茨约翰内斯·古腾堡大学医学中心)
Peking University(北京大学);The Hong Kong University of Science and Technology(香港科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
TDiR: Transformer based Diffusion for Image Restoration Tasks
Department of Computer Vision(计算机视觉系);FutureDataMinds;Electrical Engineering Dept(电气工程系);King Fahd University of Petroleum and Minerals(国王法赫德石油和矿产大学);Department of Computer Science and Software Engineering(计算机科学与软件工程系);University of Western Australia(西澳大学);School of Computing(计算学院)
WcDT: World-centric Diffusion Transformer for Traffic Scene Generation
Cardiff University(卡迪夫大学);University of Minnesota - Twin Cities(明尼苏达大学双城分校);Mississippi State University(密西西比州立大学);University of Electronic Science and Technology of China(电子科技大学);University of Toronto(多伦多大学);University of Virginia(弗吉尼亚大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselves
Northwestern Polytechnical University(西北工业大学);SGIT AI Lab, State Grid Corporation of China(国网SGIT人工智能实验室);Zhejiang University of Technology(浙江工业大学);Baidu Inc.(百度公司)
D$^2$iT: Dynamic Diffusion Transformer for Accurate Image Generation
University of Science and Technology of China(中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院)
SDiT: Semantic Region-Adaptive for Diffusion Transformers
University of Houston(德克萨斯大学奥斯汀分校);Rice University(里奇大学);Indiana University Bloomington(印第安纳大学布卢明顿分校);Argonne National Laboratory(阿贡国家实验室);National University of Singapore(新加坡国立大学)
FiT: Flexible Vision Transformer for Diffusion Model
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Tsinghua University(清华大学);Sydney University(悉尼大学);The University of Hong Kong(香港大学)