arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.18155cs.CV
收藏

ShapeMoiré:用于图像去摩尔纹的通道级形状引导网络

ShapeMoiré: Channel-Wise Shape-Guided Network for Image Demoiréing

National University of Singapore(新加坡国立大学);Beijing university of posts and telecommunications(北京邮电大学);Institute for Infocomm Research (I 2 R), A*STAR(资讯通信研究院(I2R),A*STAR);Alibaba Group(阿里巴巴集团)

Jinming Cao, Sicheng Shen, Qiu Zhou, Yifang Yin, Yangyan Li, Roger Zimmermann

AI总结:

本文针对显示器拍摄图像中跨 RGB 通道且重复出现的摩尔纹问题,提出 ShapeMoiré,利用 ShapeConv 与全局 Shape-Architecture 建模摩尔纹形状,在无推理额外开销下取得四个基准上的 SOTA 去摩尔纹性能。

摘要:

拍摄光电显示器时,由于显示器像素网格与相机传感器阵列之间的模拟信号干扰,通常会引入不希望出现的摩尔纹图案。本工作发现了现有图像去摩尔纹方法在很大程度上忽视的两个问题:1)摩尔纹图案在不同通道(RGB)之间存在差异;2)重复图案持续出现。然而,采用传统卷积(CNN)层无法解决这些问题。为此,本文提出使用我们最近提出的 Shape 概念。该概念最初用于对来自碎片化区域的一致特征进行建模,尤其是在 RGB-D 图像中同时存在相同或相似对象时。有趣的是,我们发现 Shape 信息能够有效捕捉伪影图像中的摩尔纹图案。受这一发现启发,我们提出了一种用于图像去摩尔纹的新方法 ShapeMoiré。除了在图像块级别建模形状特征外,我们进一步将其扩展到全局图像级别,并设计了一种新颖的 Shape-Architecture。因此,我们所提出的方法同时配备 ShapeConv 和 Shape-Architecture,可无缝集成到现有方法中,并且在推理过程中不会引入任何额外参数或计算开销。我们在四个广泛使用的数据集上进行了大量实验,结果表明,我们的 ShapeMoiré 取得了最先进的性能,尤其是在 PSNR 指标方面。

英文摘要:

Photographing optoelectronic displays often introduces unwanted moiré patterns due to analog signal interference between the pixel grids of the display and the camera sensor arrays. This work identifies two problems that are largely ignored by existing image demoiréing approaches: 1) moiré patterns vary across different channels (RGB); 2) repetitive patterns are constantly observed. However, employing conventional convolutional (CNN) layers cannot address these problems. Instead, this paper presents the use of our recently proposed \emph{Shape} concept. It was originally employed to model consistent features from fragmented regions, particularly when identical or similar objects coexist in an RGB-D image. Interestingly, we find that the Shape information effectively captures the moiré patterns in artifact images. Motivated by this discovery, we propose a new method, ShapeMoiré, for image demoiréing. Beyond modeling shape features at the patch-level, we further extend this to the global image-level and design a novel Shape-Architecture. Consequently, our proposed method, equipped with both ShapeConv and Shape-Architecture, can be seamlessly integrated into existing approaches without introducing any additional parameters or computation overhead during inference. We conduct extensive experiments on four widely used datasets, and the results demonstrate that our ShapeMoiré achieves state-of-the-art performance, particularly in terms of the PSNR metric.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑