HybridStitch: 像素和时间步级模型拼接用于扩散加速
HybridStitch: Pixel and Timestep Level Model Stitching for Diffusion Acceleration
AI总结：
HybridStitch通过结合大模型和小模型，实现T2I生成的高效加速，提升Stable Diffusion 3的生成速度1.83倍。
摘要：
扩散模型在文本到图像（T2I）生成应用中展现出了显著的能力。尽管生成输出先进，但它们却面临着沉重的计算开销，特别是对于包含数十亿参数的大模型。先前的工作表明，用较小的模型替代部分去噪步骤仍能保持生成质量。然而，这些方法只关注节省某些时间步的计算，忽略了单个时间步内的计算需求差异。在本工作中，我们提出了HybridStitch，一种新的T2I生成范式，将生成过程视为编辑过程。具体来说，我们引入了一个混合阶段，同时结合大模型和小模型。HybridStitch将整个图像分成两个区域：一个相对容易渲染的区域，使能够提前过渡到小模型；另一个更复杂的区域，因此需要大模型进行细化。HybridStitch利用小模型构建粗略草图，同时利用大模型编辑和细化复杂区域。根据我们的评估，HybridStitch在Stable Diffusion 3上实现了1.83倍的速度提升，比现有的混合模型方法更快。
英文摘要：
Diffusion models have demonstrated a remarkable ability in Text-to-Image (T2I) generation applications. Despite the advanced generation output, they suffer from heavy computation overhead, especially for large models that contain tens of billions of parameters. Prior work has illustrated that replacing part of the denoising steps with a smaller model still maintains the generation quality. However, these methods only focus on saving computation for some timesteps, ignoring the difference in compute demand within one timestep. In this work, we propose HybridStitch, a new T2I generation paradigm that treats generation like editing. Specifically, we introduce a hybrid stage that jointly incorporates both the large model and the small model. HybridStitch separates the entire image into two regions: one that is relatively easy to render, enabling an early transition to the smaller model, and another that is more complex and therefore requires refinement by the large model. HybridStitch employs the small model to construct a coarse sketch while exploiting the large model to edit and refine the complex regions. According to our evaluation, HybridStitch achieves 1.83$\times$ speedup on Stable Diffusion 3, which is faster than all existing mixture of model methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
