IN2OUT：使用分层判别器微调视频修复模型以实现视频外补
IN2OUT: Fine-Tuning Video Inpainting Model for Video Outpainting Using Hierarchical Discriminator
AI总结：
针对视频外补现有方法仅生成背景、直接微调修复模型易出现模糊的问题，提出分层判别器与专用外补损失，微调修复模型实现了更优的外补效果。
摘要：
视频外补面临着在扩展画面边界的同时保持与给定内容一致性的独特挑战。本文提出，可利用擅长目标流学习与重建的视频修复模型来完成外补任务，而非像现有方法那样仅生成背景。然而，直接将修复模型应用或微调至外补任务的效果不佳，往往会导致结果模糊。我们针对判别器设计开展的大量实验表明，外补微调过程中缺失的一个关键组件，是能够有效评估扩展区域感知质量的判别器。为解决这一局限，我们将对抗训练的目标拆分为全局目标与局部目标，并提出一种可同时满足两类目标的分层判别器。此外，我们还设计了一种专用的外补损失函数，该函数利用判别器的局部特征与全局特征。基于此对抗损失函数进行微调，可提升生成器生成兼具视觉吸引力与全局一致性的外补场景的能力。我们提出的方法在定量与定性指标上均优于当前最优方法。包含演示视频与代码的补充材料可在SigPort获取。
英文摘要：
Video outpainting presents a unique challenge of extending the borders while maintaining consistency with the given content. In this paper, we suggest the use of video inpainting models that excel in object flow learning and reconstruction in outpainting rather than solely generating the background as in existing methods. However, directly applying or fine-tuning inpainting models to outpainting has shown to be ineffective, often leading to blurry results. Our extensive experiments on discriminator designs reveal that a critical component missing in the outpainting fine-tuning process is a discriminator capable of effectively assessing the perceptual quality of the extended areas. To tackle this limitation, we differentiate the objectives of adversarial training into global and local goals and introduce a hierarchical discriminator that meets both objectives. Additionally, we develop a specialized outpainting loss function that leverages both local and global features of the discriminator. Fine-tuning on this adversarial loss function enhances the generator's ability to produce both visually appealing and globally coherent outpainted scenes. Our proposed method outperforms state-of-the-art methods both quantitatively and qualitatively. Supplementary materials including the demo video and the code are available in SigPort.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
