FRAG:用于扩散视频编辑的频率自适应组
FRAG: Frequency Adapting Group for Diffusion Video Editing
AI总结:
针对扩散视频编辑中因高频泄漏导致的模糊、闪烁等质量下降问题,提出无需额外训练的模型无关频率自适应组FRAG,通过保留高频成分提升质量,在TGVE、DAVIS基准验证有效。
摘要:
在视频编辑中,高质量编辑的标志在于其一致且不突兀的调整。修改在整合时必须平滑且细微,保留自然流畅度并与原始视觉效果无缝对齐。因此,我们的主要关注点是克服当前高质量编辑中的挑战,确保每次编辑在不破坏预期本质的情况下增强最终产品。然而,在最近的扩散视频编辑系统中,经常观察到模糊和闪烁等质量下降问题。我们证实,这种下降通常源于高频泄漏:扩散模型在去噪过程中无法准确合成高频成分。为此,我们设计了频率自适应组(FRAG),通过引入一个新颖的感受野分支在去噪过程中保留高频成分,从而在一致性和保真度方面提升视频质量。FRAG以模型无关的方式执行,无需额外训练,并在视频编辑基准(即TGVE、DAVIS)上验证了其有效性。
英文摘要:
In video editing, the hallmark of a quality edit lies in its consistent and unobtrusive adjustment. Modification, when integrated, must be smooth and subtle, preserving the natural flow and aligning seamlessly with the original vision. Therefore, our primary focus is on overcoming the current challenges in high quality edit to ensure that each edit enhances the final product without disrupting its intended essence. However, quality deterioration such as blurring and flickering is routinely observed in recent diffusion video editing systems. We confirm that this deterioration often stems from high-frequency leak: the diffusion model fails to accurately synthesize high-frequency components during denoising process. To this end, we devise Frequency Adapting Group (FRAG) which enhances the video quality in terms of consistency and fidelity by introducing a novel receptive field branch to preserve high-frequency components during the denoising process. FRAG is performed in a model-agnostic manner without additional training and validates the effectiveness on video editing benchmarks (i.e., TGVE, DAVIS).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。