NEC-Diff: 用于极端黑暗中感知运动的噪声鲁棒事件-RAW互补扩散
NEC-Diff: Noise-Robust Event-RAW Complementary Diffusion for Seeing Motion in Extreme Darkness
AI总结：
NEC-Diff通过结合RAW图像的线性光响应和事件的亮度变化特性，提出一种新型扩散框架，以在极低光照条件下实现高保真视觉重建。
摘要：
在极低光照条件下动态场景的高质量成像极具挑战性。光子稀缺导致严重的噪声和纹理损失，造成显著的图像退化。事件相机具有高动态范围（120 dB）和高运动敏感性，通过提供关键线索来保留细微纹理，成为传统相机的强大补充。然而，大多数现有方法侧重于从事件中恢复纹理，而忽视了图像噪声或事件本身的固有噪声，这最终阻碍了在光子匮乏条件下精确的像素重建。在本工作中，我们提出了NEC-Diff，一种新颖的基于扩散的事件-RAW混合成像框架，能够从嘈杂信号中提取可靠信息以重建精细场景结构。该框架由两个关键洞察驱动：（1）结合RAW图像的线性光响应特性与事件的亮度变化特性，建立物理驱动约束以实现鲁棒的双模态去噪；（2）基于去噪结果动态估计两种模态的信噪比（SNR），以指导自适应特征融合，从而将可靠线索注入扩散过程以实现高保真的视觉重建。此外，我们构建了REAL（Raw and Event Acquired in Low-light）数据集，提供了47,800个像素对齐的低光照RAW图像、事件和高质量参考图像，在0.001-0.8 lux光照条件下。广泛的实验表明，NEC-Diff在极端黑暗条件下具有优势。该项目可在：https://github.com/jinghan-xu/NEC-Diff 上获取。
英文摘要：
High-quality imaging of dynamic scenes in extremely low-light conditions is highly challenging. Photon scarcity induces severe noise and texture loss, causing significant image degradation. Event cameras, featuring a high dynamic range (120 dB) and high sensitivity to motion, serve as powerful complements to conventional cameras by offering crucial cues for preserving subtle textures. However, most existing approaches emphasize texture recovery from events, while paying little attention to image noise or the intrinsic noise of events themselves, which ultimately hinders accurate pixel reconstruction under photon-starved conditions. In this work, we propose NEC-Diff, a novel diffusion-based event-RAW hybrid imaging framework that extracts reliable information from heavily noisy signals to reconstruct fine scene structures. The framework is driven by two key insights: (1) combining the linear light-response property of RAW images with the brightness-change nature of events to establish a physics-driven constraint for robust dual-modal denoising; and (2) dynamically estimating the SNR of both modalities based on denoising results to guide adaptive feature fusion, thereby injecting reliable cues into the diffusion process for high-fidelity visual reconstruction. Furthermore, we construct the REAL (Raw and Event Acquired in Low-light) dataset which provides 47,800 pixel-aligned low-light RAW images, events, and high-quality references under 0.001-0.8 lux illumination. Extensive experiments demonstrate the superiority of NEC-Diff under extreme darkness. The project are available at: https://github.com/jinghan-xu/NEC-Diff.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
