arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.06351cs.CVcs.AIcs.LG
收藏

DC-DiT:通过动态分块实现视觉生成的自适应计算与弹性推理

DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking

Advanced Micro Devices Inc. (AMD)(先进微器件公司(AMD))

Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi, Mehdi Rezagholizadeh, Vikram Appia, Emad Barsoum

AI总结:

DC-DiT通过动态分块机制,自适应地压缩视觉输入,优化计算资源分配,提升生成效率与灵活性,实验显示在ImageNet生成任务中显著降低计算量并提高生成质量。

摘要:

扩散变换器依赖静态的补丁化分词,将相同的token预算分配给平滑背景、细节区域、噪声早期时间步和后期细化阶段。我们引入动态分块扩散变换器(DC-DiT),用学习的编码器-路由器-解码器架构替代固定补丁化,通过端到端的扩散训练学习动态分块机制,自适应地将2D输入压缩为更短的token序列。DC-DiT为可预测区域和噪声时间步分配更少的token,为细节区域和后期细化阶段分配更多token,从而在无监督的情况下获得有意义的空间分割和时间步适应性压缩调度。此外,路由器对保留的token提供重要性排序,使弹性推理成为可能:单个检查点可以在灵活的计算预算下进行评估,具有平滑的质量-计算权衡。此外,DC-DiT可以从预训练的DiT检查点中重新利用,并且也与正交的动态计算方法兼容。在类条件ImageNet生成任务中,DC-DiT将推理FLOPs减少高达36.8%,并相对于DiT基线提高FID高达37.8%,在模型规模、分辨率和引导设置下获得更强大的质量-计算帕累托前沿。更广泛地说,这些结果表明,自适应tokenization是使视觉生成在效率和推理灵活性方面都更优的一般机制。

英文摘要:

Diffusion Transformers rely on static patchify tokenization, assigning the same token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage refinements. We introduce the Dynamic Chunking Diffusion Transformer (DC-DiT), which replaces fixed patchification with a learned encoder-router-decoder scaffold that adaptively compresses the 2D input into a shorter token sequence through a chunking mechanism learned end-to-end with diffusion training. DC-DiT allocates fewer tokens to predictable regions and noisy timesteps, and more tokens to detailed regions and later refinement stages, yielding meaningful spatial segmentations and timestep-adaptive compression schedules without supervision. Furthermore, the router provides an importance ordering over retained tokens, enabling elastic inference: a single checkpoint can be evaluated at flexible compute budgets with a smooth quality-compute tradeoff. Additionally, DC-DiT can be upcycled from pretrained DiT checkpoints and is also compatible with orthogonal dynamic computation approaches. On class-conditional ImageNet generation, DC-DiT reduces inference FLOPs by up to 36.8% and improves FID by up to 37.8% over DiT baselines, yielding a stronger quality--compute Pareto frontier across model scales, resolutions, and guidance settings. More broadly, these results suggest that adaptive tokenization is a general mechanism for making visual generation both more efficient and more flexible at inference time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑