arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.12704cs.CV
收藏

研究现实世界视频超分辨率的权衡

Investigating Tradeoffs in Real-World Video Super-Resolution

Nanyang Technological University(南洋理工大学)

Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy

摘要:

真实世界视频超分辨率(VSR)中退化的多样性和复杂性对推理和训练提出了重要挑战。首先,虽然长期传播导致在轻度退化的情况下提高性能,但严重的野生退化可能会通过传播而被夸大,从而损害输出质量。为了平衡细节合成和伪影抑制之间的权衡,我们发现图像预清洗阶段是必不可少的,以减少传播之前的噪声和伪影。配备了精心设计的清洁模块,我们的RealBasicVSR在质量和效率方面都优于现有方法。其次,真实世界的VSR模型通常使用不同的退化来训练,以提高泛化能力,这需要增加批量大小来产生稳定的梯度。不可避免地,增加的计算负担导致各种问题,包括1)速度-性能权衡和2)批长度权衡。为了减轻第一个权衡,我们提出了一个随机降级方案,减少了40%的训练时间,而不牺牲性能。然后,我们分析了不同的训练设置,并建议在训练过程中使用更长的序列而不是更大的批次,可以更有效地使用时间信息,从而在推理过程中获得更稳定的性能。为了便于公平的比较,我们提出了新的VideoLQ数据集,它包含了大量的真实世界的低质量视频序列,包含丰富的纹理和模式。我们的数据集可以作为基准测试的共同基础。代码、模型和数据集将公开提供。

英文摘要:

The diversity and complexity of degradations in real-world video super-resolution (VSR) pose non-trivial challenges in inference and training. First, while long-term propagation leads to improved performance in cases of mild degradations, severe in-the-wild degradations could be exaggerated through propagation, impairing output quality. To balance the tradeoff between detail synthesis and artifact suppression, we found an image pre-cleaning stage indispensable to reduce noises and artifacts prior to propagation. Equipped with a carefully designed cleaning module, our RealBasicVSR outperforms existing methods in both quality and efficiency. Second, real-world VSR models are often trained with diverse degradations to improve generalizability, requiring increased batch size to produce a stable gradient. Inevitably, the increased computational burden results in various problems, including 1) speed-performance tradeoff and 2) batch-length tradeoff. To alleviate the first tradeoff, we propose a stochastic degradation scheme that reduces up to 40\% of training time without sacrificing performance. We then analyze different training settings and suggest that employing longer sequences rather than larger batches during training allows more effective uses of temporal information, leading to more stable performance during inference. To facilitate fair comparisons, we propose the new VideoLQ dataset, which contains a large variety of real-world low-quality video sequences containing rich textures and patterns. Our dataset can serve as a common ground for benchmarking. Code, models, and the dataset will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑