arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.13388cs.CVcs.ROeess.IV
收藏

PanoFlow:学习360°光流以理解周围的时间

PanoFlow: Learning 360° Optical Flow for Surrounding Temporal Understanding

Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院)

Hao Shi, Yifan Zhou, Kailun Yang, Xiaoting Yin, Ze Wang, Yaozu Ye, Zhe Yin, Shi Meng, Peng Li, Kaiwei Wang

摘要:

光流估计是自动驾驶和机器人系统中的一项基本任务,它能够在时间上解释交通场景。自动驾驶汽车显然受益于360°全景传感器提供的超宽视场(FoV)。然而,由于全景相机独特的成像过程,针对针孔图像设计的模型不能直接令人满意地推广到360°全景图像。本文提出了一种新的全景图像光流学习网络框架PanoFlow。为了克服等矩形投影在全景变换中引入的畸变,设计了一种包含径向流畸变(FDA-R)或等矩形流畸变(FDA-E)的流畸变增强(FDA)方法。我们进一步研究了全景视频的循环光流的定义和性质,并在此提出了一种循环流估计(CFE)方法,该方法利用球面图像的循环性来推断360°光流,并将大位移转换为相对小的位移。PanoFlow适用于任何现有的流量估计方法,并受益于窄FoV流量估计的进展。此外,我们创建并发布了一个基于CARLA的合成全景数据集FlowScape,以促进训练和定量分析。PanoFlow在公共OmniFlowNet和已建立的FlowScape基准上实现了最先进的性能。我们提出的方法将FlowScape上的终点误差(EPE)降低了27.3%。在OmniFlowNet上,PanoFlow实现了比最佳发布结果减少55.5%的错误。我们还通过收集车辆和公开的真实世界OmniPhotos数据集定性验证了我们的方法,表明现实世界导航应用的强大潜力和鲁棒性。代码和数据集可在https://github.com/MasterHow/PanoFlow上公开获取。

英文摘要:

Optical flow estimation is a basic task in self-driving and robotics systems, which enables to temporally interpret traffic scenes. Autonomous vehicles clearly benefit from the ultra-wide Field of View (FoV) offered by 360° panoramic sensors. However, due to the unique imaging process of panoramic cameras, models designed for pinhole images do not directly generalize satisfactorily to 360° panoramic images. In this paper, we put forward a novel network framework--PanoFlow, to learn optical flow for panoramic images. To overcome the distortions introduced by equirectangular projection in panoramic transformation, we design a Flow Distortion Augmentation (FDA) method, which contains radial flow distortion (FDA-R) or equirectangular flow distortion (FDA-E). We further look into the definition and properties of cyclic optical flow for panoramic videos, and hereby propose a Cyclic Flow Estimation (CFE) method by leveraging the cyclicity of spherical images to infer 360° optical flow and converting large displacement to relatively small displacement. PanoFlow is applicable to any existing flow estimation method and benefits from the progress of narrow-FoV flow estimation. In addition, we create and release a synthetic panoramic dataset FlowScape based on CARLA to facilitate training and quantitative analysis. PanoFlow achieves state-of-the-art performance on the public OmniFlowNet and the established FlowScape benchmarks. Our proposed approach reduces the End-Point-Error (EPE) on FlowScape by 27.3%. On OmniFlowNet, PanoFlow achieves a 55.5% error reduction from the best published result. We also qualitatively validate our method via a collection vehicle and a public real-world OmniPhotos dataset, indicating strong potential and robustness for real-world navigation applications. Code and dataset are publicly available at https://github.com/MasterHow/PanoFlow.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑