自我超越:外部特征引导是否对于加速扩散变换器训练是必不可少的?
Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?
AI总结:
本文提出SelfTranscendence方法,通过内部特征监督实现快速收敛,无需外部特征引导,在图像生成任务中表现优于现有方法。
摘要:
近期研究表明,通过外部语义特征(如DINO)引导扩散模型可以显著加速扩散变换器(DiTs)的训练。然而,使用预训练外部特征作为引导信号引入了额外依赖。我们主张DiTs实际上有能力引导自身训练,并提出SelfTranscendence方法,仅使用内部特征监督实现快速收敛。所需的内部引导特征应满足两个要求:结构清洁以帮助浅层块分离噪声与信号,语义判别以帮助浅层学习有效表示。为此,我们首先对齐DiT特征与清洁VAE潜在特征,进行短期训练(如40轮)以提升其结构表示,然后对中间特征应用分类器自由引导,增强其判别能力和语义表达性。这些在模型内部学习的丰富特征用作监督信号,引导新的DiT训练。与现有自包含方法相比,我们的方法实现了显著性能提升。它甚至在图像生成和文本到图像生成任务中,生成质量和收敛速度均超越了使用外部DINO特征的REPA方法。我们的方法源代码可在https://github.com/csslc/Self-Transcendence找到。
英文摘要:
Recent works such as REPA have shown that guiding diffusion models with external semantic features (e.g., DINO) can significantly accelerate the training of diffusion transformers (DiTs). However, the use of pretrained external features as guidance signals introduces additional dependencies. We argue that DiTs actually have the power to guide the training of themselves, and propose SelfTranscendence, an effective method that achieves fast convergence using internal feature supervision only. The desired internal guidance features should meet two requirements: structurally clean to help shallow blocks separate noise from signal, and semantically discriminative to help shallow layers learn effective representations. With this consideration, we first align the DiT features with the clean VAE latent features, a native component of latent diffusion, for a short training phase (e.g., 40 epochs) to improve their structural representations, then apply the classifier-free guidance to the intermediate features, enhancing their discriminative capability and semantic expressiveness. These enriched internal features, learned entirely within the model, are used as supervision signals to guide a new DiT training from scratch. Compared to existing self-contained methods, our approach achieves a significant performance boost. It can even surpass REPA, which uses the external DINO features as guidance, in both generation quality and convergence speed for both class-to-image and text-to-image generation tasks. The source code of our method can be found at https://github.com/csslc/Self-Transcendence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。