arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.10679cs.CV
收藏

免对齐的Raw视频去摩尔纹

Alignment-free Raw Video Demoireing

Shuning Xu, Xina Liu, Binbin Song, Xiangyu Chen, Qiubo Chen, Jiantao Zhou

AI总结:

本文提出了一种免对齐的Raw视频去摩尔纹网络DemMamba,通过频率辅助的时空Mamba模块有效建模时空关系,在PSNR上超越现有方法1.6 dB。

摘要:

视频去摩尔纹旨在去除屏幕内容捕获过程中产生的不必要干扰图案,在保持时间一致性的同时恢复无伪影的帧。现有视频去摩尔纹方法通常利用精心设计的对齐模块来估计帧间运动以利用时间信息;然而,这些模块通常很复杂且计算要求高。同时,最近的工作表明,使用raw数据作为输入显著增强了去摩尔纹性能。基于这一见解,本文引入了一种带有频率辅助时空Mamba(DemMamba)的新型免对齐raw视频去摩尔纹网络。它结合了顺序排列的Spatial Mamba Blocks(SMB)和Temporal Mamba Blocks(TMB),以有效建模raw视频去摩尔纹中的帧间和帧内关系。SMB采用多方向扫描机制,结合可学习的频率压缩器,有效区分各种方向和频率上的干扰图案,从而减少伪影、锐化边缘并忠实重建纹理。同时,TMB通过在时间序列上执行双向扫描并集成通道注意力技术来增强时间一致性,促进改进的时间信息融合。大量实验表明,DemMamba在PSNR上超越了最先进的方法1.6 dB,并提供了令人满意的视觉体验。

英文摘要:

Video demoireing aims to remove undesirable interference patterns that arise during the capture of screen content, restoring artifact-free frames while maintaining temporal consistency. Existing video demoireing methods typically utilize carefully designed alignment modules to estimate inter-frame motion for leveraging temporal information; however, these modules are often complex and computationally demanding. Meanwhile, recent works indicate that using raw data as input significantly enhances demoireing performance. Building on this insight, this paper introduces a novel alignment-free raw video demoireing network with frequency-assisted spatio-temporal Mamba (DemMamba). It incorporates sequentially arranged Spatial Mamba Blocks (SMB) and Temporal Mamba Blocks (TMB) to effectively model the inter- and intra-relationships in raw video demoireing. The SMB employs a multi-directional scanning mechanism coupled with a learnable frequency compressor to effectively differentiate interference patterns across various orientations and frequencies, resulting in reduced artifacts, sharper edges, and faithful texture reconstruction. Concurrently, the TMB enhances temporal consistency by performing bidirectional scanning across the temporal sequences and integrating channel attention techniques, facilitating improved temporal information fusion. Extensive experiments demonstrate that DemMamba surpasses state-of-the-art methods by 1.6 dB in PSNR, and also delivers a satisfactory visual experience.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑