拖动你的GAN：在生成图像流形上进行基于点的交互式操作
Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
AI总结：
提出DragGAN，通过基于特征的运动监督和点跟踪，在生成图像流形上实现交互式拖动点来精确控制图像姿态、形状、表情和布局，优于现有方法。
摘要：
合成满足用户需求的视觉内容通常需要对生成对象的姿态、形状、表情和布局进行灵活且精确的控制。现有方法通过手动标注的训练数据或先验3D模型来获得生成对抗网络（GAN）的可控性，这些方法往往缺乏灵活性、精确性和通用性。在本工作中，我们研究了一种强大但探索较少的方式来控制GAN，即“拖动”图像中的任意点，以用户交互的方式精确到达目标点，如图1所示。为实现这一目标，我们提出了DragGAN，它包含两个主要组件：1）基于特征的运动监督，驱动手柄点向目标位置移动；2）一种新的点跟踪方法，利用判别性生成器特征来持续定位手柄点的位置。通过DragGAN，任何人都可以精确控制像素的移动方向来变形图像，从而操作各种类别（如动物、汽车、人类、风景等）的姿态、形状、表情和布局。由于这些操作是在GAN的学习生成图像流形上执行的，即使面对具有挑战性的场景，如幻觉遮挡内容和遵循物体刚性的形状变形，它们也倾向于产生逼真的输出。定性和定量比较均表明，DragGAN在图像操作和点跟踪任务上优于先前方法。我们还通过GAN反演展示了真实图像的操作。
英文摘要：
Synthesizing visual content that meets users' needs often requires flexible and precise controllability of the pose, shape, expression, and layout of the generated objects. Existing approaches gain controllability of generative adversarial networks (GANs) via manually annotated training data or a prior 3D model, which often lack flexibility, precision, and generality. In this work, we study a powerful yet much less explored way of controlling GANs, that is, to "drag" any points of the image to precisely reach target points in a user-interactive manner, as shown in Fig.1. To achieve this, we propose DragGAN, which consists of two main components: 1) a feature-based motion supervision that drives the handle point to move towards the target position, and 2) a new point tracking approach that leverages the discriminative generator features to keep localizing the position of the handle points. Through DragGAN, anyone can deform an image with precise control over where pixels go, thus manipulating the pose, shape, expression, and layout of diverse categories such as animals, cars, humans, landscapes, etc. As these manipulations are performed on the learned generative image manifold of a GAN, they tend to produce realistic outputs even for challenging scenarios such as hallucinating occluded content and deforming shapes that consistently follow the object's rigidity. Both qualitative and quantitative comparisons demonstrate the advantage of DragGAN over prior approaches in the tasks of image manipulation and point tracking. We also showcase the manipulation of real images through GAN inversion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
