PRE-Mamba:用于超高频率事件相机去雨的4D状态空间模型
PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining
AI总结:
PRE-Mamba提出基于点的4D状态空间事件相机去雨框架,通过双时间尺度表示、时空解耦融合和多尺度状态空间模型,以0.26M参数在EventRain-27K上实现0.95 SR、0.91 NR的高效去雨,并泛化至雪况。
摘要:
事件相机在高时间分辨率和动态范围方面表现出色,但在雨天条件下会遭受密集噪声的困扰。现有的事件去雨方法在时间精度、去雨效果和计算效率之间面临权衡。在本文中,我们提出了PRE-Mamba,一种新颖的基于点的事件相机去雨框架,该框架充分利用了原始事件和雨的时空特性。我们的框架引入了一种4D事件云表示,该表示集成了双时间尺度以保持高时间精度;一个时空解耦与融合模块(STDF),通过实现时间和空间信息的浅层解耦与交互来增强去雨能力;以及一个多尺度状态空间模型(MS3M),以线性计算复杂度在双时间和多空间尺度上捕获更深的雨动态。通过频域正则化的增强,PRE-Mamba在EventRain-27K(一个包含标记合成和真实世界序列的综合数据集)上仅用0.26M参数就实现了卓越的性能(0.95 SR,0.91 NR,和0.4秒/M事件)。此外,我们的方法在不同雨强度、视角甚至雪况下都能很好地泛化。
英文摘要:
Event cameras excel in high temporal resolution and dynamic range but suffer from dense noise in rainy conditions. Existing event deraining methods face trade-offs between temporal precision, deraining effectiveness, and computational efficiency. In this paper, we propose PRE-Mamba, a novel point-based event camera deraining framework that fully exploits the spatiotemporal characteristics of raw event and rain. Our framework introduces a 4D event cloud representation that integrates dual temporal scales to preserve high temporal precision, a Spatio-Temporal Decoupling and Fusion module (STDF) that enhances deraining capability by enabling shallow decoupling and interaction of temporal and spatial information, and a Multi-Scale State Space Model (MS3M) that captures deeper rain dynamics across dual-temporal and multi-spatial scales with linear computational complexity. Enhanced by frequency-domain regularization, PRE-Mamba achieves superior performance (0.95 SR, 0.91 NR, and 0.4s/M events) with only 0.26M parameters on EventRain-27K, a comprehensive dataset with labeled synthetic and real-world sequences. Moreover, our method generalizes well across varying rain intensities, viewpoints, and even snowy conditions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。