FreeMask：重新思考注意力掩码在零样本视频编辑中的重要性
FreeMask: Rethinking the Importance of Attention Masks for Zero-Shot Video Editing
AI总结：
本文提出FreeMask方法，通过掩码匹配代价（MMC）度量选择最优交叉注意力掩码，并改进掩码融合机制，无需控制辅助或参数微调即可提升零样本视频编辑的语义保真度、时间一致性和编辑质量。
摘要：
文本到视频扩散模型已经取得了显著进展。在这些基础模型能够生成时间一致视频的能力驱动下，利用这些基础模型进行零样本视频编辑的研究迅速扩展。为了提高编辑质量，视频编辑中经常采用结构控制。在这些技术中，交叉注意力掩码控制因其有效性和高效性而脱颖而出。然而，当交叉注意力掩码被简单地应用于视频编辑时，它们可能引入模糊和闪烁等伪影。我们的实验揭示了一个以往视频编辑研究中被忽视的关键因素：交叉注意力掩码并非始终清晰，而是随模型结构和去噪时间步而变化。为了解决这一问题，我们提出了度量指标掩码匹配代价（MMC）来量化这种变异性，并提出了FreeMask，一种针对特定视频编辑任务选择最优掩码的方法。利用MMC选择的掩码，我们进一步改进了综合注意力特征中的掩码融合机制，例如时间、交叉和自注意力模块。我们的方法可以无缝集成到现有的零样本视频编辑框架中，并获得更好的性能，无需控制辅助或参数微调，同时能够通过掩码精度控制实现未编辑语义布局的自适应解耦。大量实验表明，与最先进的方法相比，FreeMask在语义保真度、时间一致性和编辑质量方面均取得了优越表现。
英文摘要：
Text-to-video diffusion models have made remarkable advancements. Driven by their ability to generate temporally coherent videos, research on zero-shot video editing using these fundamental models has expanded rapidly. To enhance editing quality, structural controls are frequently employed in video editing. Among these techniques, cross-attention mask control stands out for its effectiveness and efficiency. However, when cross-attention masks are naively applied to video editing, they can introduce artifacts such as blurring and flickering. Our experiments uncover a critical factor overlooked in previous video editing research: cross-attention masks are not consistently clear but vary with model structure and denoising timestep. To address this issue, we propose the metric Mask Matching Cost (MMC) that quantifies this variability and propose FreeMask, a method for selecting optimal masks tailored to specific video editing tasks. Using MMC-selected masks, we further improve the masked fusion mechanism within comprehensive attention features, e.g., temp, cross, and self-attention modules. Our approach can be seamlessly integrated into existing zero-shot video editing frameworks with better performance, requiring no control assistance or parameter fine-tuning but enabling adaptive decoupling of unedited semantic layouts with mask precision control. Extensive experiments demonstrate that FreeMask achieves superior semantic fidelity, temporal consistency, and editing quality compared to state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
