arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.09427cs.CVcs.LGcs.RO
收藏

基于时空神经网络和动态视觉传感器的快速运动理解

Fast Motion Understanding with Spatiotemporal Neural Networks and Dynamic Vision Sensors

Samsung AI Center NY(三星纽约人工智能中心)

Anthony Bisulco, Fernando Cladera Ojeda, Volkan Isler, Daniel D. Lee

摘要:

本文提出了一种基于动态视觉传感器(DVS)的高速运动推理系统。作为一个代表性的场景,我们考虑的情况下,一个机器人在休息的反应,一个小的,快速接近的物体速度高于15米/秒。由于传统的图像传感器在典型的帧速率下观察这样的对象仅几帧,因此估计潜在的运动对标准计算机视觉系统和算法提出了相当大的挑战。在本文中,我们提出了一种方法的动机是如何动物,如昆虫解决这个问题与他们相对简单的视觉系统。我们的解决方案从DVS中获取事件流,并首先使用一组跨多个时间尺度的因果指数过滤器对时间事件进行编码。我们将这些滤波器与卷积神经网络(CNN)相结合,以有效地提取相关的时空特征。组合网络学习输出物体碰撞的预期时间以及离散化极坐标网格上的预测碰撞点。这些关键估计是由网络以最小的延迟计算的,以便对传入的对象做出适当的反应。我们强调了我们的系统的结果,以23.4m/s的速度移动的玩具飞镖,在$θ$中的误差为24.73°,平均离散半径预测误差为18.4mm,碰撞预测误差的中值为25.03%。

英文摘要:

This paper presents a Dynamic Vision Sensor (DVS) based system for reasoning about high speed motion. As a representative scenario, we consider the case of a robot at rest reacting to a small, fast approaching object at speeds higher than 15m/s. Since conventional image sensors at typical frame rates observe such an object for only a few frames, estimating the underlying motion presents a considerable challenge for standard computer vision systems and algorithms. In this paper we present a method motivated by how animals such as insects solve this problem with their relatively simple vision systems. Our solution takes the event stream from a DVS and first encodes the temporal events with a set of causal exponential filters across multiple time scales. We couple these filters with a Convolutional Neural Network (CNN) to efficiently extract relevant spatiotemporal features. The combined network learns to output both the expected time to collision of the object, as well as the predicted collision point on a discretized polar grid. These critical estimates are computed with minimal delay by the network in order to react appropriately to the incoming object. We highlight the results of our system to a toy dart moving at 23.4m/s with a 24.73° error in $θ$, 18.4mm average discretized radius prediction error, and 25.03% median time to collision prediction error.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑