FlowTurbo:通过速度精炼器迈向基于流的实时图像生成
FlowTurbo: Towards Real-time Flow-Based Image Generation with Velocity Refiner
AI总结:
本文提出 FlowTurbo 框架,通过引入轻量级速度精炼器、伪校正器和样本感知编译技术,在保持或提升采样质量的同时加速基于流的生成模型,实现了实时图像生成并达到新的最优水平。
摘要:
基于扩散模型在视觉生成领域的成功,基于流的模型重新成为另一类重要的生成模型家族,在视觉质量和推理速度方面均取得了具有竞争力甚至更优的性能。通过流匹配学习速度场,基于流的模型倾向于产生更直的采样轨迹,这在采样过程中是有利的。然而,与已开发出成熟快速采样器的扩散模型不同,基于流的生成模型的高效采样鲜少被探索。在本文中,我们提出了一个名为 FlowTurbo 的框架,以在提升采样质量的同时加速基于流模型的采样。我们的主要观察是,基于流模型中速度预测器的输出在采样过程中会变得稳定,从而可以通过轻量级速度精炼器来估计速度。此外,我们引入了包括伪校正器和样本感知编译在内的几种技术,以进一步减少推理时间。由于 FlowTurbo 不改变多步采样范式,它可以有效地应用于诸如图像编辑、图像修复等各种任务。通过将 FlowTurbo 集成到不同的基于流的模型中,我们在类条件生成上获得了 53.1%$\sim$58.3% 的加速比,在文本到图像生成上获得了 29.8%$\sim$38.5% 的加速比。值得注意的是,FlowTurbo 在 ImageNet 上以 100 (ms / img) 的速度达到了 2.12 的 FID,并以 38 (ms / img) 的速度达到了 3.93 的 FID,实现了实时图像生成并确立了新的最优水平。代码可在 https://github.com/shiml20/FlowTurbo 获取。
英文摘要:
Building on the success of diffusion models in visual generation, flow-based models reemerge as another prominent family of generative models that have achieved competitive or better performance in terms of both visual quality and inference speed. By learning the velocity field through flow-matching, flow-based models tend to produce a straighter sampling trajectory, which is advantageous during the sampling process. However, unlike diffusion models for which fast samplers are well-developed, efficient sampling of flow-based generative models has been rarely explored. In this paper, we propose a framework called FlowTurbo to accelerate the sampling of flow-based models while still enhancing the sampling quality. Our primary observation is that the velocity predictor's outputs in the flow-based models will become stable during the sampling, enabling the estimation of velocity via a lightweight velocity refiner. Additionally, we introduce several techniques including a pseudo corrector and sample-aware compilation to further reduce inference time. Since FlowTurbo does not change the multi-step sampling paradigm, it can be effectively applied for various tasks such as image editing, inpainting, etc. By integrating FlowTurbo into different flow-based models, we obtain an acceleration ratio of 53.1%$\sim$58.3% on class-conditional generation and 29.8%$\sim$38.5% on text-to-image generation. Notably, FlowTurbo reaches an FID of 2.12 on ImageNet with 100 (ms / img) and FID of 3.93 with 38 (ms / img), achieving the real-time image generation and establishing the new state-of-the-art. Code is available at https://github.com/shiml20/FlowTurbo.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。