SpikeReveal：利用脉冲流从真实模糊输入中解锁时间序列
SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams
AI总结：
针对现有监督式脉冲引导运动去模糊方法泛化性差的问题，提出首个自监督级联框架SpikeReveal，结合知识蒸馏与再模糊损失生成高质量清晰序列，在真实与合成数据集上验证了更优的泛化性能。
摘要：
从模糊输入中重建清晰图像序列对于加深我们对拍摄场景的理解至关重要，但由于图像中嵌入的时间特征有限，这一任务面临着重大挑战。采样率高达40000Hz的脉冲相机（Spike camera）已被证明能有效捕捉运动特征，有助于解决这一不适定问题。尽管如此，现有方法都属于监督学习范式，当应用于与合成训练数据域存在差异的真实场景时，性能会出现显著下降。此外，重建图像的质量受限于基于运动分析插值生成的图像，这类图像本质上与真实场景存在差异，影响了这些方法在真实高速场景中的泛化能力。为应对这些挑战，我们提出了首个用于脉冲引导运动去模糊任务的自监督框架。我们的方法首先构建了一个脉冲引导去模糊模型，探究了脉冲流、模糊图像及其对应清晰序列之间的理论关系。随后，我们开发了一个自监督级联框架，以缓解去模糊模型中遇到的脉冲噪声和空间分辨率不匹配问题。借助知识蒸馏和再模糊损失，我们进一步设计了一个轻量级去模糊网络，用于生成与原始输入在亮度和纹理上保持一致的高质量序列。在我们构建的含脉冲的真实世界数据集和合成数据集上开展的定量与定性实验，验证了所提框架具备优异的泛化能力。我们的代码、数据和训练好的模型将发布于https://github.com/chenkang455/S-SDM。
英文摘要：
Reconstructing a sequence of sharp images from the blurry input is crucial for enhancing our insights into the captured scene and poses a significant challenge due to the limited temporal features embedded in the image. Spike cameras, sampling at rates up to 40,000 Hz, have proven effective in capturing motion features and beneficial for solving this ill-posed problem. Nonetheless, existing methods fall into the supervised learning paradigm, which suffers from notable performance degradation when applied to real-world scenarios that diverge from the synthetic training data domain. Moreover, the quality of reconstructed images is capped by the generated images based on motion analysis interpolation, which inherently differs from the actual scene, affecting the generalization ability of these methods in real high-speed scenarios. To address these challenges, we propose the first self-supervised framework for the task of spike-guided motion deblurring. Our approach begins with the formulation of a spike-guided deblurring model that explores the theoretical relationships among spike streams, blurry images, and their corresponding sharp sequences. We subsequently develop a self-supervised cascaded framework to alleviate the issues of spike noise and spatial-resolution mismatching encountered in the deblurring model. With knowledge distillation and re-blurring loss, we further design a lightweight deblur network to generate high-quality sequences with brightness and texture consistency with the original input. Quantitative and qualitative experiments conducted on our real-world and synthetic datasets with spikes validate the superior generalization of the proposed framework. Our code, data and trained models will be available at \url{https://github.com/chenkang455/S-SDM}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
