在向量场中引导 Rectified Flow 模型以实现可控图像生成
Steering Rectified Flow Models in the Vector Field for Controlled Image Generation
AI总结：
本文提出 FlowChef 框架，利用 RFMs 向量场以无梯度方式引导去噪轨迹，统一解决分类器引导、线性逆问题和图像编辑，并在性能、内存与速度上达到新 SOTA。
摘要：
扩散模型（DMs）在照片级真实感、图像编辑和求解逆问题方面表现出色，这得益于无分类器引导和图像反演技术。然而，Rectified Flow 模型（RFMs）在这些任务上仍未得到充分研究。现有基于 DM 的方法通常需要额外训练，难以泛化到预训练潜空间模型，性能不佳，并且由于通过 ODE 求解器和反演过程进行大量反向传播而需要大量计算资源。在本工作中，我们首先从理论和实证上理解 RFMs 的向量场动力学，以高效引导去噪轨迹。研究发现表明，我们可以以确定性且无需梯度的方式在向量场中导航。利用这一特性，我们提出了 FlowChef，它借助向量场并通过梯度跳过来引导去噪轨迹，从而完成可控图像生成任务。FlowChef 是一个统一的可控图像生成框架，首次同时处理分类器引导、线性逆问题和图像编辑，而无需额外训练、反演或密集反向传播。最后，我们进行了广泛评估，结果表明 FlowChef 在性能、内存和时间需求方面显著优于基线，取得了新的最先进结果。项目页面：https://flowchef.github.io。
英文摘要：
Diffusion models (DMs) excel in photorealism, image editing, and solving inverse problems, aided by classifier-free guidance and image inversion techniques. However, rectified flow models (RFMs) remain underexplored for these tasks. Existing DM-based methods often require additional training, lack generalization to pretrained latent models, underperform, and demand significant computational resources due to extensive backpropagation through ODE solvers and inversion processes. In this work, we first develop a theoretical and empirical understanding of the vector field dynamics of RFMs in efficiently guiding the denoising trajectory. Our findings reveal that we can navigate the vector field in a deterministic and gradient-free manner. Utilizing this property, we propose FlowChef, which leverages the vector field to steer the denoising trajectory for controlled image generation tasks, facilitated by gradient skipping. FlowChef is a unified framework for controlled image generation that, for the first time, simultaneously addresses classifier guidance, linear inverse problems, and image editing without the need for extra training, inversion, or intensive backpropagation. Finally, we perform extensive evaluations and show that FlowChef significantly outperforms baselines in terms of performance, memory, and time requirements, achieving new state-of-the-art results. Project Page: \url{https://flowchef.github.io}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
