AdaFlow：通过自适应注意力瘦身与关键帧选择实现高效长视频编辑
AdaFlow: Efficient Long Video Editing via Adaptive Attention Slimming And Keyframe Selection
AI总结：
针对文本驱动长视频编辑的内存开销问题，提出无需训练的AdaFlow方法，通过自适应注意力瘦身压缩KV序列、自适应关键帧选择提升质量，实现单GPU超1k帧编辑，长度达对比方法十倍，并构建LongV-EVAL基准。
摘要：
尽管取得了巨大进展，文本驱动的长视频编辑仍因过高的内存开销而极具挑战性。尽管近期研究将该任务简化为关键帧转换与插值生成两步流程，但逐token的关键帧转换仍限制了视频长度的上限。本文提出一种新颖且无需训练的高效长视频编辑方法AdaFlow。首先，我们发现视频帧的所有token对关键帧转换的重要性并非均等，据此提出自适应注意力瘦身方案，压缩KV序列，使可转换的关键帧数量提升一个数量级。此外，配备自适应关键帧选择方案，挑选代表性帧进行联合编辑，进一步提升生成质量。凭借这些创新设计，AdaFlow可在单次推理中实现分钟级高质量长视频编辑，如在单张A800 GPU上处理超1k帧，长度约为TokenFlow等对比方法的十倍。为验证AdaFlow，我们构建了带高质量标注的长视频编辑新基准LongV-EVAL。代码发布于：https://github.com/jidantang55/AdaFlow。
英文摘要：
Despite great progress, text-driven long video editing is still notoriously challenging mainly due to excessive memory overhead. Although recent efforts have simplified this task into a two-step process of keyframe translation and interpolation generation, the token-wise keyframe translation still plagues the upper limit of video length. In this paper, we propose a novel and training-free approach towards efficient and effective long video editing, termed AdaFlow. We first reveal that not all tokens of video frames hold equal importance for keyframe translation, based on which we propose an Adaptive Attention Slimming scheme for AdaFlow to squeeze the $KV$ sequence, thus increasing the number of keyframes for translations by an order of magnitude. In addition, an Adaptive Keyframe Selection scheme is also equipped to select the representative frames for joint editing, further improving generation quality. With these innovative designs, AdaFlow achieves high-quality long video editing of minutes in one inference, i.e., more than 1$k$ frames on one A800 GPU, which is about ten times longer than the compared methods, e.g., TokenFlow. To validate AdaFlow, we also build a new benchmark for long video editing with high-quality annotations, termed LongV-EVAL. Our code is released at: https://github.com/jidantang55/AdaFlow.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
