MotionEditor：基于内容感知扩散的视频运动编辑
MotionEditor: Editing Video Motion via Content-Aware Diffusion
AI总结：
本文提出MotionEditor，通过内容感知运动适配器、双分支架构及高保真注意力注入和骨架对齐算法，在保留原始主角外观与背景的同时实现视频运动编辑。
摘要：
现有的基于扩散的视频编辑模型在随时间编辑源视频属性方面取得了显著进展，但在操纵运动信息的同时保留原始主角外观和背景方面仍面临困难。为了解决这一问题，我们提出了MotionEditor，一种用于视频运动编辑的扩散模型。MotionEditor将一种新颖的内容感知运动适配器引入ControlNet，以捕捉时间运动对应关系。虽然ControlNet能够基于骨架姿态直接生成，但在修改反转噪声中的源运动时会遇到挑战，因为噪声（源）与条件（参考）之间存在矛盾信号。我们的适配器通过引入源内容来无缝传递适配后的控制信号，从而对ControlNet形成补充。此外，我们构建了一个双分支架构（重建分支和编辑分支），并采用高保真注意力注入机制促进分支交互。该机制使编辑分支能够以解耦方式从重建分支查询键和值，从而使编辑分支保留原始背景和主角外观。我们还提出了一种骨架对齐算法，以解决姿态尺寸和位置上的差异。实验从定性和定量两方面证明了MotionEditor在运动编辑方面的出色能力。
英文摘要：
Existing diffusion-based video editing models have made gorgeous advances for editing attributes of a source video over time but struggle to manipulate the motion information while preserving the original protagonist's appearance and background. To address this, we propose MotionEditor, a diffusion model for video motion editing. MotionEditor incorporates a novel content-aware motion adapter into ControlNet to capture temporal motion correspondence. While ControlNet enables direct generation based on skeleton poses, it encounters challenges when modifying the source motion in the inverted noise due to contradictory signals between the noise (source) and the condition (reference). Our adapter complements ControlNet by involving source content to transfer adapted control signals seamlessly. Further, we build up a two-branch architecture (a reconstruction branch and an editing branch) with a high-fidelity attention injection mechanism facilitating branch interaction. This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance. We also propose a skeleton alignment algorithm to address the discrepancies in pose size and position. Experiments demonstrate the promising motion editing ability of MotionEditor, both qualitatively and quantitatively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
