Drag-A-Video：基于点交互的非刚性视频编辑
Drag-A-Video: Non-rigid Video Editing with Point-based Interaction
AI总结：
针对交互式非刚性视频编辑问题，提出基于扩散的 Drag-A-Video，通过点集传播、视频级运动监督、潜在偏移和时间一致点跟踪实现精确且跨帧一致的拖拽操控。
摘要：
视频编辑是一项具有挑战性的任务，需要在空间和时间两个维度上操控视频。现有的视频编辑方法主要关注改变视频中对象的外观或风格，同时保持其结构不变。然而，目前还没有方法允许用户交互式地“拖拽”第一帧上实例的任意点，使其精确到达目标点，并使其他帧产生一致的形变。本文提出一种新的基于扩散的交互式基于点的视频操控方法，称为 Drag-A-Video。该方法允许用户在输入视频的第一帧上点击成对的手柄点和目标点，并提供掩码。随后，方法将这些输入转换为点集，并在各帧之间传播这些点集。为了精确修改视频内容，我们采用一种新的视频级运动监督来更新视频特征，并引入潜在偏移，以在多个去噪时间步实现这种更新。我们提出一个时间一致的点跟踪模块，用于协调手柄点集中各点的运动。我们在多种视频上展示了该方法的有效性和灵活性。本工作网站位于：https://drag-a-video.github.io/。
英文摘要：
Video editing is a challenging task that requires manipulating videos on both the spatial and temporal dimensions. Existing methods for video editing mainly focus on changing the appearance or style of the objects in the video, while keeping their structures unchanged. However, there is no existing method that allows users to interactively ``drag'' any points of instances on the first frame to precisely reach the target points with other frames consistently deformed. In this paper, we propose a new diffusion-based method for interactive point-based video manipulation, called Drag-A-Video. Our method allows users to click pairs of handle points and target points as well as masks on the first frame of an input video. Then, our method transforms the inputs into point sets and propagates these sets across frames. To precisely modify the contents of the video, we employ a new video-level motion supervision to update the features of the video and introduce the latent offsets to achieve this update at multiple denoising timesteps. We propose a temporal-consistent point tracking module to coordinate the movement of the points in the handle point sets. We demonstrate the effectiveness and flexibility of our method on various videos. The website of our work is available here: https://drag-a-video.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
