UniEdit：面向视频运动与外观编辑的统一免调优框架
UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing
AI总结：
本文提出免调优框架 UniEdit，利用预训练文本到视频生成器及运动参考、重建分支与时空注意力注入，统一支持视频运动和外观编辑，并在实验中超过现有方法。
摘要：
文本引导视频编辑的最新进展已在外观编辑（例如风格化）方面展示出令人鼓舞的成果。然而，时间维度上的视频运动编辑（例如从进食变为挥手）虽体现了视频编辑与图像编辑的区别，却仍未得到充分研究。在本文中，我们提出 UniEdit，这是一个免调优框架，通过在“先反演、后生成”的框架内利用预训练文本到视频生成器的能力，同时支持视频运动编辑和外观编辑。为了在保留源视频内容的同时实现运动编辑，基于时间自注意力层和空间自注意力层分别编码帧间依赖与帧内依赖这一认识，我们引入辅助运动参考分支和重建分支，以分别生成文本引导的运动特征和源特征。随后，所得特征通过时间和空间自注意力层注入主编辑路径。大量实验表明，UniEdit 涵盖视频运动编辑及多种外观编辑场景，并超越了最先进方法。我们的代码将公开发布。
英文摘要：
Recent advances in text-guided video editing have showcased promising results in appearance editing (e.g., stylization). However, video motion editing in the temporal dimension (e.g., from eating to waving), which distinguishes video editing from image editing, is underexplored. In this work, we present UniEdit, a tuning-free framework that supports both video motion and appearance editing by harnessing the power of a pre-trained text-to-video generator within an inversion-then-generation framework. To realize motion editing while preserving source video content, based on the insights that temporal and spatial self-attention layers encode inter-frame and intra-frame dependency respectively, we introduce auxiliary motion-reference and reconstruction branches to produce text-guided motion and source features respectively. The obtained features are then injected into the main editing path via temporal and spatial self-attention layers. Extensive experiments demonstrate that UniEdit covers video motion editing and various appearance editing scenarios, and surpasses the state-of-the-art methods. Our code will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
