MaskINT：基于插值式非自回归掩码Transformer的视频编辑
MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformers
AI总结：
本文提出MaskINT，将文本视频编辑分为零样本关键帧编辑与非自回归掩码Transformer帧插值两阶段，在保持与扩散模型相当性能的同时显著提升推理效率。
摘要：
生成式人工智能的最新进展显著增强了图像和视频编辑能力，尤其是在文本提示控制方面。当前最先进的方法主要依赖扩散模型来完成这些任务。然而，基于扩散的方法计算需求巨大，通常需要大规模成对数据集进行训练，因此难以在实际应用中部署。为解决这些问题，本文将基于文本的视频编辑任务分解为两个阶段。首先，我们利用预训练的文本到图像扩散模型以零样本方式同时编辑少量关键帧。其次，我们引入一个名为MaskINT的高效模型，该模型建立在非自回归掩码生成Transformer之上，专门用于在已编辑关键帧之间进行帧插值，并利用中间帧的结构引导。实验结果表明，我们的MaskINT取得了与基于扩散的方法相当的性能，同时显著缩短了推理时间。本研究为基于文本的视频编辑提供了一种实用解决方案，并展示了非自回归掩码生成Transformer在该领域的潜力。
英文摘要：
Recent advances in generative AI have significantly enhanced image and video editing, particularly in the context of text prompt control. State-of-the-art approaches predominantly rely on diffusion models to accomplish these tasks. However, the computational demands of diffusion-based methods are substantial, often necessitating large-scale paired datasets for training, and therefore challenging the deployment in real applications. To address these issues, this paper breaks down the text-based video editing task into two stages. First, we leverage an pre-trained text-to-image diffusion model to simultaneously edit few keyframes in an zero-shot way. Second, we introduce an efficient model called MaskINT, which is built on non-autoregressive masked generative transformers and specializes in frame interpolation between the edited keyframes, using the structural guidance from intermediate frames. Experimental results suggest that our MaskINT achieves comparable performance with diffusion-based methodologies, while significantly improve the inference time. This research offers a practical solution for text-based video editing and showcases the potential of non-autoregressive masked generative transformers in this domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
