arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.04745cs.CV

面向视频超分辨率的协作反馈判别传播

Collaborative Feedback Discriminative Propagation for Video Super-Resolution

Nanjing University of Science and Technology(南京理工大学)

Hao Li, Xiang Chen, Jiangxin Dong, Jinhui Tang, Jinshan Pan

AI总结:

本文提出协作反馈判别(CFD)方法,通过判别对齐校正(DAC)和协作反馈传播(CFP)模块校正对齐伪影并融合多时间步时空信息,在保持低复杂度下提升视频超分辨率性能。

摘要:

现有视频超分辨率(VSR)方法的关键成功主要源于对空间和时间信息的探索,这通常通过带有对齐模块的循环传播模块来实现。然而,不准确的对齐通常会导致对齐后的特征带有显著伪影,这些伪影会在传播过程中不断累积,从而影响视频恢复。此外,传播模块仅向前或向后传播同一时间步的特征,在复杂运动或遮挡情况下可能失效,限制了其高质量帧恢复的性能。为了解决这些问题,我们提出了一种协作反馈判别(CFD)方法,用于校正不准确的对齐特征并建模长程空间和时间信息,以实现更好的视频重建。具体而言,我们开发了一种判别对齐校正(DAC)方法,自适应地探索信息并减少由不准确对齐引起的伪影的影响。然后,我们提出了一个协作反馈传播(CFP)模块,该模块采用反馈和门控机制,同时从向前和向后传播中更好地探索不同时间步特征的空间和时间信息。最后,我们将所提出的DAC和CFP嵌入到常用的VSR网络中,以验证我们方法的有效性。在多个基准上的定量和定性实验表明,我们的方法能够在保持较低模型复杂度的同时提升现有VSR模型的性能。源代码和预训练模型将在\url{https://github.com/House-Leo/CFDVSR}上公开。

英文摘要:

The key success of existing video super-resolution (VSR) methods stems mainly from exploring spatial and temporal information, which is usually achieved by a recurrent propagation module with an alignment module. However, inaccurate alignment usually leads to aligned features with significant artifacts, which will be accumulated during propagation and thus affect video restoration. Moreover, propagation modules only propagate the same timestep features forward or backward that may fail in case of complex motion or occlusion, limiting their performance for high-quality frame restoration. To address these issues, we propose a collaborative feedback discriminative (CFD) method to correct inaccurate aligned features and model long -range spatial and temporal information for better video reconstruction. In detail, we develop a discriminative alignment correction (DAC) method to adaptively explore information and reduce the influences of the artifacts caused by inaccurate alignment. Then, we propose a collaborative feedback propagation (CFP) module that employs feedback and gating mechanisms to better explore spatial and temporal information of different timestep features from forward and backward propagation simultaneously. Finally, we embed the proposed DAC and CFP into commonly used VSR networks to verify the effectiveness of our method. Quantitative and qualitative experiments on several benchmarks demonstrate that our method can improve the performance of existing VSR models while maintaining a lower model complexity. The source code and pre-trained models will be available at \url{https://github.com/House-Leo/CFDVSR}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑