arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.12265cs.CVcs.AI
收藏

3D- FlowNet:基于事件的3D表示光流估计

3D-FlowNet: Event-based optical flow estimation with 3D representation

Nantes Université(南特大学);Ecole Centrale de Nantes(南特中央理工学院);CNRS LS2N(法国国家科学研究中心南特数字科学实验室)

Haixin Sun, Minh-Quan Dao, Vincent Fremont

摘要:

基于事件的相机可以超越基于帧的相机的限制,用于重要任务,例如在低照明条件下自动驾驶汽车导航期间的高速运动检测。事件摄像机的高时间分辨率和高动态范围使其能够在快速运动和极端光线情况下工作。然而,传统的计算机视觉方法,例如深度神经网络,并不适合于处理事件数据,因为它们是异步和离散的。此外,传统的事件数据的二维编码表示方法,牺牲了时间分辨率。在本文中,我们首先通过将其扩展到三维来改进2D编码表示,以更好地保留事件的时间分布。然后,我们提出了3D-FlowNet,一种新的网络架构,可以处理的3D输入表示和输出光流估计根据新的编码方法。采用自监督训练策略来补偿基于事件的相机的标记数据集的缺乏。最后,建议的网络进行训练和评估与多车辆立体事件摄像机(MVSEC)数据集。结果表明,我们的3D-FlowNet优于最先进的方法,训练时间更少(30与Spike-FlowNet的100相比)。

英文摘要:

Event-based cameras can overpass frame-based cameras limitations for important tasks such as high-speed motion detection during self-driving cars navigation in low illumination conditions. The event cameras' high temporal resolution and high dynamic range, allow them to work in fast motion and extreme light scenarios. However, conventional computer vision methods, such as Deep Neural Networks, are not well adapted to work with event data as they are asynchronous and discrete. Moreover, the traditional 2D-encoding representation methods for event data, sacrifice the time resolution. In this paper, we first improve the 2D-encoding representation by expanding it into three dimensions to better preserve the temporal distribution of the events. We then propose 3D-FlowNet, a novel network architecture that can process the 3D input representation and output optical flow estimations according to the new encoding methods. A self-supervised training strategy is adopted to compensate the lack of labeled datasets for the event-based camera. Finally, the proposed network is trained and evaluated with the Multi-Vehicle Stereo Event Camera (MVSEC) dataset. The results show that our 3D-FlowNet outperforms state-of-the-art approaches with less training epoch (30 compared to 100 of Spike-FlowNet).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑