基于3D点轨迹的生成视频运动编辑
Generative Video Motion Editing with 3D Point Tracks
AI总结：
本文提出基于3D点轨迹的视频生成框架，实现摄像机与物体运动的联合编辑，通过稀疏对应关系保持时空一致性，提升视频编辑的精确度和创作潜力。
摘要：
摄像机和物体的运动是视频叙事的核心。然而，精确编辑这些捕捉到的运动仍是一个重大挑战，尤其是在复杂的物体运动情况下。当前的运动控制图像到视频（I2V）方法往往缺乏一致视频编辑的全场景上下文，而视频到视频（V2V）方法只能提供视角变化或基本物体平移，但对细粒度物体运动控制有限。我们提出了一种基于轨迹的V2V框架，能够联合编辑摄像机和物体运动。我们通过将视频生成模型条件化于源视频和配对的3D点轨迹（代表源和目标运动）来实现这一点。这些3D轨迹建立了稀疏对应关系，将丰富的上下文从源视频转移到新运动，同时保持时空一致性。关键的是，与2D轨迹相比，3D轨迹提供了显式的深度线索，使模型能够解决深度顺序并处理遮挡，以实现精确的运动编辑。在合成和真实数据上训练两阶段，我们的模型支持多种运动编辑，包括联合摄像机/物体操控、运动转移和非刚性变形，解锁了视频编辑的新创作潜力。
英文摘要：
Camera and object motions are central to a video's narrative. However, precisely editing these captured motions remains a significant challenge, especially under complex object movements. Current motion-controlled image-to-video (I2V) approaches often lack full-scene context for consistent video editing, while video-to-video (V2V) methods provide viewpoint changes or basic object translation, but offer limited control over fine-grained object motion. We present a track-conditioned V2V framework that enables joint editing of camera and object motion. We achieve this by conditioning a video generation model on a source video and paired 3D point tracks representing source and target motions. These 3D tracks establish sparse correspondences that transfer rich context from the source video to new motions while preserving spatiotemporal coherence. Crucially, compared to 2D tracks, 3D tracks provide explicit depth cues, allowing the model to resolve depth order and handle occlusions for precise motion editing. Trained in two stages on synthetic and real data, our model supports diverse motion edits, including joint camera/object manipulation, motion transfer, and non-rigid deformation, unlocking new creative potential in video editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
