基于递归时空融合的多级视频去噪算法
Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion
摘要：
近年来，基于深度学习的去噪方法以较大的计算复杂度为代价取得了无与伦比的性能。在这项工作中，我们提出了一种高效的多级视频去噪算法，称为EMVD，大大降低了复杂度，同时保持甚至提高了性能。首先，融合阶段通过递归组合视频中所有过去的帧来降低噪声。然后，去噪阶段去除融合帧中的噪声。最后，细化阶段恢复去噪帧中丢失的高频。所有阶段都在通过可学习和可逆线性算子获得的变换域表示上操作，这同时提高了模型的准确性并降低了模型的复杂性。最终输出上的单个损失足以成功收敛，因此使得EMVD易于训练。真实原始数据上的实验表明，EMVD优于最先进的复杂性受到限制时，甚至仍然具有竞争力的方法，其复杂性是几个数量级更高。此外，EMVD的低复杂性和存储器要求使得能够在移动设备中的商业SoC上进行实时视频降噪。
英文摘要：
In recent years, denoising methods based on deep learning have achieved unparalleled performance at the cost of large computational complexity. In this work, we propose an Efficient Multi-stage Video Denoising algorithm, called EMVD, to drastically reduce the complexity while maintaining or even improving the performance. First, a fusion stage reduces the noise through a recursive combination of all past frames in the video. Then, a denoising stage removes the noise in the fused frame. Finally, a refinement stage restores the missing high frequency in the denoised frame. All stages operate on a transform-domain representation obtained by learnable and invertible linear operators which simultaneously increase accuracy and decrease complexity of the model. A single loss on the final output is sufficient for successful convergence, hence making EMVD easy to train. Experiments on real raw data demonstrate that EMVD outperforms the state of the art when complexity is constrained, and even remains competitive against methods whose complexities are several orders of magnitude higher. Further, the low complexity and memory requirements of EMVD enable real-time video denoising on commercial SoC in mobile devices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
