VideoSwap：利用交互式语义点对应实现定制化视频主体替换
VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence
AI总结：
针对扩散视频编辑难以处理形状变化的问题，本文提出 VideoSwap 框架，用少量可交互语义点对齐运动并改变主体形状，在真实视频主体替换上达到 SOTA。
摘要：
当前基于扩散模型的视频编辑主要通过利用各种密集对应关系来关注结构保持编辑，以确保时间一致性和运动对齐。然而，当目标编辑涉及形状变化时，这些方法往往效果不佳。为开展支持形状变化的视频编辑，我们在本工作中探索定制化视频主体替换，目标是将源视频中的主要主体替换为具有不同身份且可能具有不同形状的目标主体。与以往依赖密集对应关系的方法不同，我们提出了 VideoSwap 框架，该框架利用语义点对应关系；其灵感来自我们的观察：只需少量语义点即可对齐主体的运动轨迹并修改其形状。我们还引入了多种用户点交互（例如删除点和拖拏点），以处理各种语义点对应关系。大量实验表明，该方法在多种真实世界视频上取得了最先进的视频主体替换结果。
英文摘要：
Current diffusion-based video editing primarily focuses on structure-preserved editing by utilizing various dense correspondences to ensure temporal consistency and motion alignment. However, these approaches are often ineffective when the target edit involves a shape change. To embark on video editing with shape change, we explore customized video subject swapping in this work, where we aim to replace the main subject in a source video with a target subject having a distinct identity and potentially different shape. In contrast to previous methods that rely on dense correspondences, we introduce the VideoSwap framework that exploits semantic point correspondences, inspired by our observation that only a small number of semantic points are necessary to align the subject's motion trajectory and modify its shape. We also introduce various user-point interactions (\eg, removing points and dragging points) to address various semantic point correspondence. Extensive experiments demonstrate state-of-the-art video subject swapping results across a variety of real-world videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
