MPDiT：一种多块全局到局部变换器架构用于高效的流匹配和扩散模型
MPDiT: Multi-Patch Global-to-Local Transformer Architecture For Efficient Flow Matching and Diffusion Model
AI总结：
MPDiT通过多块变换器架构在扩散和流匹配模型中实现高效计算，减少50%的GFLOPs消耗同时保持生成性能，改进时间与类别嵌入设计加速训练收敛。
摘要：
Transformer架构，特别是扩散变换器（DiTs），因其性能优于卷积UNet而在扩散和流匹配模型中广泛应用。然而，DiTs的各向同性设计在每个块中处理相同数量的块化标记，导致训练过程计算较重。本文引入了一种多块变换器设计，早期块使用较大块捕获粗略全局上下文，后期块使用较小块细化局部细节。这种分层设计可将计算成本降低高达50%的GFLOPs，同时保持良好的生成性能。此外，我们还提出了改进的时间和类别嵌入设计，以加速训练收敛。在ImageNet数据集上的广泛实验验证了我们的架构选择的有效性。代码已发布在：https://github.com/quandao10/MPDiT
英文摘要：
Transformer architectures, particularly Diffusion Transformers (DiTs), have become widely used in diffusion and flow-matching models due to their strong performance compared to convolutional UNets. However, the isotropic design of DiTs processes the same number of patchified tokens in every block, leading to relatively heavy computation during training process. In this work, we introduce a multi-patch transformer design in which early blocks operate on larger patches to capture coarse global context, while later blocks use smaller patches to refine local details. This hierarchical design could reduces computational cost by up to 50% in GFLOPs while achieving good generative performance. In addition, we also propose improved designs for time and class embeddings that accelerate training convergence. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our architectural choices. Code is released at: https://github.com/quandao10/MPDiT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
