arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.09317cs.CVcs.LGeess.IV
收藏

神经网络如何估计光流?一项受神经心理学启发的研究

How Do Neural Networks Estimate Optical Flow? A Neuropsychology-Inspired Study

Delft University of Technology(代尔夫特理工大学)

D. B. de Jong, F. Paredes-Vallés, G. C. H. E. de Croon

摘要:

端到端训练的卷积神经网络在光流估计方面取得了突破。最新的进展集中在通过改进架构和在公开可用的MPI-Sintel数据集上设置新的基准来改进光流估计。相反,在本文中,我们研究了深度神经网络如何估计光流。更好地理解这些网络的功能对于(i)评估它们对未知输入的泛化能力以及(ii)提出改进其性能的建议非常重要。对于我们的研究,我们专注于FlowNetS,因为它是用于光流估计的编码器-解码器神经网络的原型。此外,我们使用的过滤器识别方法,发挥了重要作用,在揭示运动过滤器存在于动物大脑中的神经心理学研究。该方法表明,在最深层的FlowNetS的过滤器是敏感的各种运动模式。我们不仅发现了动物大脑中的平移过滤器,而且由于人工神经网络中更容易测量,我们甚至揭示了扩张,旋转和遮挡过滤器。此外,我们发现在网络的细化部分和感知填充过程中发生在哺乳动物初级视觉皮层的相似之处。

英文摘要:

End-to-end trained convolutional neural networks have led to a breakthrough in optical flow estimation. The most recent advances focus on improving the optical flow estimation by improving the architecture and setting a new benchmark on the publicly available MPI-Sintel dataset. Instead, in this article, we investigate how deep neural networks estimate optical flow. A better understanding of how these networks function is important for (i) assessing their generalization capabilities to unseen inputs, and (ii) suggesting changes to improve their performance. For our investigation, we focus on FlowNetS, as it is the prototype of an encoder-decoder neural network for optical flow estimation. Furthermore, we use a filter identification method that has played a major role in uncovering the motion filters present in animal brains in neuropsychological research. The method shows that the filters in the deepest layer of FlowNetS are sensitive to a variety of motion patterns. Not only do we find translation filters, as demonstrated in animal brains, but thanks to the easier measurements in artificial neural networks, we even unveil dilation, rotation, and occlusion filters. Furthermore, we find similarities in the refinement part of the network and the perceptual filling-in process which occurs in the mammal primary visual cortex.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑