基于通道与空间调制的原始屏幕图像与视频去摩尔纹方法
Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations
AI总结：
本文针对原始域屏幕图像与视频中的摩尔纹问题，提出结合颜色分离分支与通道、空间调制的去摩尔纹网络，并构建首个原始视频去摩尔纹数据集RawVDemoiré，取得最先进性能。
摘要：
通过智能手机相机拍摄屏幕内容已成为信息共享的常见方式。然而，这些图像和视频常常因相机滤波阵列与数字显示网格之间的频率混叠而产生摩尔纹，导致质量下降。我们观察到，原始域中的摩尔纹比sRGB域中的更简单，并且原始色彩通道中的摩尔纹具有不同特性。因此，我们提出了一种面向原始输入的图像和视频去摩尔纹网络。我们引入了一个颜色分离特征分支，并通过通道与空间调制将其与传统的特征混合分支融合。具体而言，通道调制利用调制后的颜色分离特征来增强颜色混合特征；空间调制则利用具有大感受野的特征来调制具有小感受野的特征。此外，我们构建了第一个良好对齐的原始视频去摩尔纹（RawVDemoiré）数据集，并提出了一种通过插入交替图案的高效时间对齐方法。实验表明，我们的方法在图像和视频去摩尔纹任务上均达到了最先进的性能。我们已在https://github.com/tju-chengyijia/VD_raw发布代码和数据集。
英文摘要：
Capturing screen contents by smartphone cameras has become a common way for information sharing. However, these images and videos are often degraded by moiré patterns, which are caused by frequency aliasing between the camera filter array and digital display grids. We observe that the moiré patterns in raw domain is simpler than those in sRGB domain, and the moiré patterns in raw color channels have different properties. Therefore, we propose an image and video demoiréing network tailored for raw inputs. We introduce a color-separated feature branch, and it is fused with the traditional feature-mixed branch via channel and spatial modulations. Specifically, the channel modulation utilizes modulated color-separated features to enhance the color-mixed features. The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field. In addition, we build the first well-aligned raw video demoiréing (RawVDemoiré) dataset and propose an efficient temporal alignment method by inserting alternating patterns. Experiments demonstrate that our method achieves state-of-the-art performance for both image and video demoriéing. We have released the code and dataset in https://github.com/tju-chengyijia/VD_raw.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
