DragVideo：交互式拖拽式视频编辑
DragVideo: Interactive Drag-style Video Editing
AI总结：
针对视频编辑中用户控制不精确、易产生失真伪影及时空不一致的问题，提出DragVideo框架，采用拖拽式潜在优化与LoRA及互自注意力机制，实现直观、忠实且时空一致的视频编辑。
摘要：
视频生成模型已展现出生成逼真视频的卓越能力。然而，如何精确控制（或编辑）视频仍然是一个艰巨的挑战。主要问题在于：1）如何在编辑中实现直接且精确的用户控制；2）如何在不产生难看的失真和伪影的情况下执行诸如改变形状、表情和布局等编辑操作；以及3）如何在编辑后保持视频的时空一致性。为解决上述问题，我们提出了DragVideo，一个通用的拖拽式视频编辑框架。受DragGAN启发，DragVideo通过提出拖拽式视频潜在优化方法来解决第1）和第2）个问题，该方法通过视频级拖拽目标函数，根据拖拽指令更新带噪声的视频潜在表示，从而实现所需的控制。我们通过将视频扩散模型与样本特定的LoRA和互自注意力机制集成到DragVideo中，修正了第3）个问题，以确保编辑结果在时空上保持一致。我们还提供了一系列拖拽式视频编辑的测试示例，并在广泛的具有挑战性的编辑任务（如运动、骨骼编辑等）上进行了大量实验，突显了DragVideo能够以直观、忠实于用户意图的方式编辑视频，失真和伪影几乎不可察觉，同时保持时空一致性。虽然传统的基于提示的视频编辑无法实现前两者，而直接应用图像拖拽编辑在最后一点上失败，但DragVideo的通用性和普适性得到了强调。GitHub链接：https://github.com/RickySkywalker/DragVideo-Official。
英文摘要：
Video generation models have shown their superior ability to generate photo-realistic video. However, how to accurately control (or edit) the video remains a formidable challenge. The main issues are: 1) how to perform direct and accurate user control in editing; 2) how to execute editings like changing shape, expression, and layout without unsightly distortion and artifacts to the edited content; and 3) how to maintain spatio-temporal consistency of video after editing. To address the above issues, we propose DragVideo, a general drag-style video editing framework. Inspired by DragGAN, DragVideo addresses issues 1) and 2) by proposing the drag-style video latent optimization method which gives desired control by updating noisy video latent according to drag instructions through video-level drag objective function. We amend issue 3) by integrating the video diffusion model with sample-specific LoRA and Mutual Self-Attention in DragVideo to ensure the edited result is spatio-temporally consistent. We also present a series of testing examples for drag-style video editing and conduct extensive experiments across a wide array of challenging editing tasks, such as motion, skeleton editing, etc, underscoring DragVideo can edit video in an intuitive, faithful to the user's intention manner, with nearly unnoticeable distortion and artifacts, while maintaining spatio-temporal consistency. While traditional prompt-based video editing fails to do the former two and directly applying image drag editing fails in the last, DragVideo's versatility and generality are emphasized. Github link: https://github.com/RickySkywalker/DragVideo-Official.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
