Motion Guidance：基于可微运动估计器的扩散模型图像编辑
Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators
AI总结：
针对扩散模型难以精确编辑图像中物体布局、姿态等的问题，提出零样本运动引导技术，通过光流网络梯度引导扩散采样，实现指定运动场的高质量图像运动编辑。
摘要：
扩散模型能够基于文本描述生成效果出色的图像，这类模型的扩展版本可支持用户在较粗粒度上编辑图像。然而，利用扩散模型精确编辑图像中物体的布局、位置、姿态和形状仍然颇具难度。针对这一问题，我们提出motion guidance（运动引导），这是一种零样本技术，允许用户指定密集、复杂的运动场，用以指示图像中每个像素应移动到的位置。运动引导的工作原理是通过现成的光流网络传递梯度，以此引导扩散采样过程。具体而言，我们设计了一种引导损失，它既能促使采样结果具备由光流网络估计的预期运动，又能保持与源图像的视觉相似性。通过同时从扩散模型中采样并引导采样结果维持较低的引导损失，我们可以得到经运动编辑的图像。实验证明，我们的技术适用于复杂运动场景，能够对真实图像和生成图像实现高质量编辑。
英文摘要：
Diffusion models are capable of generating impressive images conditioned on text descriptions, and extensions of these models allow users to edit images at a relatively coarse scale. However, the ability to precisely edit the layout, position, pose, and shape of objects in images with diffusion models is still difficult. To this end, we propose motion guidance, a zero-shot technique that allows a user to specify dense, complex motion fields that indicate where each pixel in an image should move. Motion guidance works by steering the diffusion sampling process with the gradients through an off-the-shelf optical flow network. Specifically, we design a guidance loss that encourages the sample to have the desired motion, as estimated by a flow network, while also being visually similar to the source image. By simultaneously sampling from a diffusion model and guiding the sample to have low guidance loss, we can obtain a motion-edited image. We demonstrate that our technique works on complex motions and produces high quality edits of real and generated images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
