arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.06898cs.CVcs.RO
收藏

EV-FlowNet:基于事件的摄像机自监督光流估计

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

University of Pennsylvania(宾夕法尼亚大学)

Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, Kostas Daniilidis

摘要:

基于事件的相机在基于帧的相机受到影响的各种情况下(诸如高速运动和高动态范围场景)已经显示出很大的前景。然而,开发用于事件测量的算法需要一类新的手工算法。深度学习在为视觉社区中的许多问题提供无模型解决方案方面取得了巨大成功,但现有网络的开发考虑到了基于帧的图像,并且不存在用于监督训练的图像所具有的丰富的事件标记数据。针对这些问题,我们提出了EV-FlowNet,这是一种新型的自监督深度学习管道,用于基于事件的相机的光流估计。特别是,我们引入了一个给定的事件流,这是一个自我监督的神经网络作为唯一的输入馈入基于图像的表示。在给定来自网络的估计流量的情况下,与事件同时从同一相机捕获的对应灰度图像然后被用作监督信号以在训练时间提供损失函数。我们表明,由此产生的网络是能够准确地预测光流的事件,只有在各种不同的场景,与基于图像的网络的性能竞争。该方法不仅可以精确估计密集光流,而且还提供了一个框架,其他自监督的方法转移到基于事件的域。

英文摘要:

Event-based cameras have shown great promise in a variety of situations where frame based cameras suffer, such as high speed motions and high dynamic range scenes. However, developing algorithms for event measurements requires a new class of hand crafted algorithms. Deep learning has shown great success in providing model free solutions to many problems in the vision community, but existing networks have been developed with frame based images in mind, and there does not exist the wealth of labeled data for events as there does for images for supervised training. To these points, we present EV-FlowNet, a novel self-supervised deep learning pipeline for optical flow estimation for event based cameras. In particular, we introduce an image based representation of a given event stream, which is fed into a self-supervised neural network as the sole input. The corresponding grayscale images captured from the same camera at the same time as the events are then used as a supervisory signal to provide a loss function at training time, given the estimated flow from the network. We show that the resulting network is able to accurately predict optical flow from events only in a variety of different scenes, with performance competitive to image based networks. This method not only allows for accurate estimation of dense optical flow, but also provides a framework for the transfer of other self-supervised methods to the event-based domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑