Magic Fixup: 通过观看动态视频简化照片编辑
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos
AI总结：
提出一种生成模型，利用视频作为监督，将粗略编辑的图像转换为遵循布局的逼真输出，同时转移细节并处理照明和交互等二阶效应。
摘要：
我们提出了一种生成模型，给定一张粗略编辑过的图像，该模型能够合成一张遵循指定布局的逼真输出。我们的方法从原始图像中转移精细细节，并保持其各部分的身份特征。同时，它使输出适应新布局所定义的照明和上下文。我们的关键洞察是，视频是此任务的有力监督来源：物体和相机运动提供了关于世界如何随视角、照明和物理交互变化的大量观测。我们构建了一个图像数据集，其中每个样本是从同一视频中随机选择的时间间隔提取的源帧和目标帧对。我们使用两种运动模型将源帧向目标帧扭曲，以模拟预期的测试时用户编辑。我们从预训练的扩散模型开始，监督我们的模型将扭曲图像转换为真实目标。我们的模型设计明确支持从源帧到生成图像的精细细节转移，同时紧密遵循用户指定的布局。我们表明，通过使用简单的分割和粗略的2D操作，我们可以合成忠实于用户输入的逼真编辑，同时处理诸如编辑对象之间的照明和谐和物理交互等二阶效应。
英文摘要：
We propose a generative model that, given a coarsely edited image, synthesizes a photorealistic output that follows the prescribed layout. Our method transfers fine details from the original image and preserve the identity of its parts. Yet, it adapts it to the lighting and context defined by the new layout. Our key insight is that videos are a powerful source of supervision for this task: objects and camera motions provide many observations of how the world changes with viewpoint, lighting, and physical interactions. We construct an image dataset in which each sample is a pair of source and target frames extracted from the same video at randomly chosen time intervals. We warp the source frame toward the target using two motion models that mimic the expected test-time user edits. We supervise our model to translate the warped image into the ground truth, starting from a pretrained diffusion model. Our model design explicitly enables fine detail transfer from the source frame to the generated image, while closely following the user-specified layout. We show that by using simple segmentations and coarse 2D manipulations, we can synthesize a photorealistic edit faithful to the user's input while addressing second-order effects like harmonizing the lighting and physical interactions between edited objects.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
