SDformerFlow：用于基于事件的光流估计的时空 Swin Spikeformer
SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimation
AI总结：
本文提出 STTFlowNet 与全脉冲化 SDformerFlow，首次将 swin spikeformer 用于事件相机稠密光流估计，并在 DSEC、MVSEC 上达到 SNN 方法最先进水平且显著降低功耗。
摘要：
事件相机产生异步且稀疏的事件流，用于捕捉光强度的变化。与传统的基于帧的相机相比，它们具有显著优势，例如更高的动态范围和极快的数据速率，因此特别适用于涉及快速运动或挑战性光照条件的场景。脉冲神经网络（SNN）具有类似的异步和稀疏特性，非常适合处理来自事件相机的数据。受 transformer 和脉冲驱动 transformer（spikeformer）在其他计算机视觉任务中潜力的启发，我们提出了两种用于事件相机快速、鲁棒光流估计的解决方案：STTFlowNet 和 SDformerFlow。STTFlowNet 采用 U 形人工神经网络（ANN）架构，并配备时空移位窗口自注意力（swin）transformer 编码器；而 SDformerFlow 提出了其完全脉冲化的对应方案，融合了 swin spikeformer 编码器。此外，我们还给出了采用不同神经元模型的两个脉冲版本变体。我们的工作首次利用 spikeformer 进行稠密光流估计。我们使用监督学习对所有模型进行端到端训练。结果表明，所提方法在 DSEC 和 MVSEC 数据集上均取得了基于 SNN 的事件光流方法中的最先进性能，并且与等效 ANN 相比显著降低了功耗。
英文摘要：
Event cameras generate asynchronous and sparse event streams capturing changes in light intensity. They offer significant advantages over conventional frame-based cameras, such as a higher dynamic range and an extremely faster data rate, making them particularly useful in scenarios involving fast motion or challenging lighting conditions. Spiking neural networks (SNNs) share similar asynchronous and sparse characteristics and are well-suited for processing data from event cameras. Inspired by the potential of transformers and spike-driven transformers (spikeformers) in other computer vision tasks, we propose two solutions for fast and robust optical flow estimation for event cameras: STTFlowNet and SDformerFlow. STTFlowNet adopts a U-shaped artificial neural network (ANN) architecture with spatiotemporal shifted window self-attention (swin) transformer encoders, while SDformerFlow presents its fully spiking counterpart, incorporating swin spikeformer encoders. Furthermore, we present two variants of the spiking version with different neuron models. Our work is the first to make use of spikeformers for dense optical flow estimation. We conduct end-to-end training for all models using supervised learning. Our results yield state-of-the-art performance among SNN-based event optical flow methods on both the DSEC and MVSEC datasets, and show significant reduction in power consumption compared to the equivalent ANNs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
