VDFP:基于闪烁带先验的视频去闪烁
VDFP: Video Deflickering with Flicker-banding Priors
AI总结:
本文提出VDFP,一种基于闪烁带先验的视频去闪烁框架,通过构建DeViD数据集和引入DFM和CPP模块,有效解决屏幕捕捉中的带状伪影问题,实验表明其在去闪烁效果和时空一致性方面优于现有方法。
摘要:
使用智能手机捕捉数字屏幕时,由于硬件同步不匹配,经常会产生严重的带状伪影。现有的视频修复方法难以处理这些结构化、周期性的亮度波动,通常导致残留伪影或过度平滑的纹理。我们首先构建了DeViD数据集,以应对可用数据集不足的问题。然后我们提出了VDFP(Video Deflickering with Flicker-banding Priors),一种新颖的感知引导生成框架。首先,我们引入了一种基于滚动快门机制的退化场建模(DFM),能够合成复杂的多带状场景。其次,我们提出了空间-时间连续先验感知(CPP)。不同于传统的二元分割,该模块通过闪烁感知的均方误差(FA-MSE)进行优化,以捕捉亮度过渡。通过零初始化增强的输入层,我们的模型保留了预训练的生成先验以及空间-时间先验感知。广泛的实验表明,VDFP在去闪烁效果和时空一致性方面显著优于其他方法,能够高效消除复杂的带状伪影并保留高保真的空间细节。我们的数据集和代码将在https://github.com/ZhiyiZZhou/VDFP上发布。
英文摘要:
Capturing digital screens with smartphones frequently induces severe banding due to hardware synchronization mismatches. Existing video restoration methods struggle with these structured, periodic luminance fluctuations, often resulting in residual artifacts or over-smoothed textures. We firstly construct DeViD, a real-world dataset in various scenes to deal with the lack of available datasets. Then we propose VDFP (Video Deflickering with Flicker-banding Priors), a novel perception-guided generation framework. First, we introduce a Degradation Field Modeling Based on Rolling Shutter Mechanism (DFM) capable of synthesizing complex multi-banding scenarios. Second, we present a spatial-temporal continuous prior perception (CPP). Unlike traditional binary segmentation, this module is optimized via a Flicker-Aware Mean Squared Error (FA-MSE) to capture the luminance transitions. By zero-initializing an augmented input layer, our model preserves pre-trained generative priors as well as spatial-temporal prior perception. Extensive experiments demonstrate that VDFP significantly outperforms other methods, eliminating complex banding with high-fidelity spatial details and temporal consistency. Our dataset and code will be released at https://github.com/ZhiyiZZhou/VDFP.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。