TV-LiVE：通过层信息活力利用实现免训练、文本引导的视频编辑
TV-LiVE: Training-Free, Text-Guided Video Editing via Layer Informed Vitality Exploitation
AI总结：
本文提出免训练文本引导框架 TV-LiVE，通过识别并利用与 RoPE 相关的关键层注入 key/value 特征，并提取新增对象掩码，实现高质量对象添加与非刚性视频编辑。
摘要：
随着基于扩散的视频生成模型快速发展，视频编辑受到越来越多关注。在这些进展中，人们对更易使用、更可控的视频编辑形式（如基于提示的编辑）需求不断增长。以往研究主要集中于风格迁移、背景替换、对象替换和属性修改等任务，同时保持源视频的内容结构。然而，更复杂的任务，包括添加新对象和非刚性变换，仍相对未被充分探索。本文提出 TV-LiVE，这是一种通过层信息活力利用（Layer-informed Vitality Exploitation）实现的免训练、文本引导视频编辑框架。我们通过经验识别出视频生成模型中对生成输出质量有显著影响的关键层。值得注意的是，这些层与旋转位置嵌入（Rotary Position Embeddings，RoPE）密切相关。基于这一观察，我们的方法在层活力引导下，选择性地将源模型的 key 和 value 特征注入目标模型的相应层，从而同时支持对象添加和非刚性视频编辑。对于对象添加，我们进一步识别突出层，以提取与新添加的目标提示相对应的掩码区域。我们发现，从突出层提取的掩码能够忠实地指示待编辑区域。实验结果表明，TV-LiVE 在对象添加和非刚性视频编辑两方面均优于现有方法。项目页面：https://emjay73.github.io/TV_LiVE/
英文摘要：
Video editing has garnered increasing attention alongside the rapid progress of diffusion-based video generation models. As part of these advancements, there is a growing demand for more accessible and controllable forms of video editing, such as prompt-based editing. Previous studies have primarily focused on tasks such as style transfer, background replacement, object substitution, and attribute modification, while maintaining the content structure of the source video. However, more complex tasks, including the addition of novel objects and nonrigid transformations, remain relatively unexplored. In this paper, we present TV-LiVE, a Training-free and text-guided Video editing framework via Layerinformed Vitality Exploitation. We empirically identify vital layers within the video generation model that significantly influence the quality of generated outputs. Notably, these layers are closely associated with Rotary Position Embeddings (RoPE). Based on this observation, our method enables both object addition and non-rigid video editing by selectively injecting key and value features from the source model into the corresponding layers of the target model guided by the layer vitality. For object addition, we further identify prominent layers to extract the mask regions corresponding to the newly added target prompt. We found that the extracted masks from the prominent layers faithfully indicate the region to be edited. Experimental results demonstrate that TV-LiVE outperforms existing approaches for both object addition and non-rigid video editing. Project Page: https://emjay73.github.io/TV_LiVE/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
