MotionDiff：基于流辅助多视图扩散的免训练零样本交互式运动编辑
MotionDiff: Training-free Zero-shot Interactive Motion Editing via Flow-assisted Multi-view Diffusion
AI总结：
针对现有运动编辑方法难以处理复杂运动和多视图一致性的问题，提出免训练零样本的MotionDiff方法，利用光流和多视图扩散实现高质量交互式运动编辑，无需重新训练。
摘要：
生成模型已经取得了显著的进展，能够生成高质量的内容。然而，由于生成模型在输出上固有的不确定性，使用生成模型进行可控编辑仍然具有挑战性。这一挑战在涉及空间信息处理的运动编辑中尤为突出。虽然一些基于物理的生成方法尝试实现运动编辑，但它们通常处理的是具有简单运动（如平移和拖拽）的单视图图像。这些方法难以处理复杂的旋转和拉伸运动，并确保多视图一致性，往往需要资源密集型的重新训练。为了解决这些挑战，我们提出了MotionDiff，一种免训练的零样本扩散方法，利用光流进行复杂的多视图运动编辑。具体来说，给定一个静态场景，用户可以交互式地选择感兴趣的对象以添加运动先验。所提出的点运动学模型（PKM）在多视图光流估计阶段（MFES）估计相应的多视图光流。随后，在多视图运动扩散阶段（MMDS），通过解耦的运动表示利用这些光流生成多视图运动结果。大量实验表明，MotionDiff在实现高质量的多视图一致运动结果方面优于其他基于物理的生成式运动编辑方法。值得注意的是，MotionDiff不需要重新训练，使用户能够方便地将其适应于各种下游任务。
英文摘要：
Generative models have made remarkable advancements and are capable of producing high-quality content. However, performing controllable editing with generative models remains challenging, due to their inherent uncertainty in outputs. This challenge is praticularly pronounced in motion editing, which involves the processing of spatial information. While some physics-based generative methods have attempted to implement motion editing, they typically operate on single-view images with simple motions, such as translation and dragging. These methods struggle to handle complex rotation and stretching motions and ensure multi-view consistency, often necessitating resource-intensive retraining. To address these challenges, we propose MotionDiff, a training-free zero-shot diffusion method that leverages optical flow for complex multi-view motion editing. Specifically, given a static scene, users can interactively select objects of interest to add motion priors. The proposed Point Kinematic Model (PKM) then estimates corresponding multi-view optical flows during the Multi-view Flow Estimation Stage (MFES). Subsequently, these optical flows are utilized to generate multi-view motion results through decoupled motion representation in the Multi-view Motion Diffusion Stage (MMDS). Extensive experiments demonstrate that MotionDiff outperforms other physics-based generative motion editing methods in achieving high-quality multi-view consistent motion results. Notably, MotionDiff does not require retraining, enabling users to conveniently adapt it for various down-stream tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
