DiTPainter：基于扩散Transformer的高效视频修复
DiTPainter: Efficient Video Inpainting with Diffusion Transformers
AI总结：
针对现有视频修复算法易模糊不一致、预训练DiT模型参数量大耗时久的问题，提出从头训练的高效DiT端到端视频修复模型DiTPainter，可处理任意长度视频，在质量和时空一致性上优于现有算法。
摘要：
现有许多视频修复算法利用光流构建对应映射图，再通过映射将相邻帧的像素传播到缺失区域。尽管这种传播机制具备有效性，但在处理不准确光流或大掩码时，可能会出现模糊和不一致问题。近期，扩散Transformer（DiT）作为一项革命性技术应用于视频生成任务，但用于视频生成的预训练DiT模型参数量庞大，应用于视频修复任务非常耗时。本文提出DiTPainter——一种基于扩散Transformer（DiT）的端到端视频修复模型。DiTPainter采用专为视频修复设计的高效Transformer网络，从头开始训练，而非基于任何大型预训练模型初始化。它可处理任意长度的视频，能以可接受的时间成本应用于视频去字幕和视频补全任务。实验表明，DiTPainter在质量和时空一致性上均优于现有视频修复算法。
英文摘要：
Many existing video inpainting algorithms utilize optical flows to construct the corresponding maps and then propagate pixels from adjacent frames to missing areas by mapping. Despite the effectiveness of the propagation mechanism, they might encounter blurry and inconsistencies when dealing with inaccurate optical flows or large masks. Recently, Diffusion Transformer (DiT) has emerged as a revolutionary technique for video generation tasks. However, pretrained DiT models for video generation all contain a large amount of parameters, which makes it very time consuming to apply to video inpainting tasks. In this paper, we present DiTPainter, an end-to-end video inpainting model based on Diffusion Transformer (DiT). DiTPainter uses an efficient transformer network designed for video inpainting, which is trained from scratch instead of initializing from any large pretrained models. DiTPainter can address videos with arbitrary lengths and can be applied to video decaptioning and video completion tasks with an acceptable time cost. Experiments show that DiTPainter outperforms existing video inpainting algorithms with higher quality and better spatial-temporal consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
