Optical flow estimation is a basic task in self-driving and robotics systems, which enables to temporally interpret traffic scenes. Autonomous vehicles clearly benefit from the ultra-wide Field of View (FoV) offered by 360° panoramic sensors. However, due to the unique imaging process of panoramic cameras, models designed for pinhole images do not directly generalize satisfactorily to 360° panoramic images. In this paper, we put forward a novel network framework--PanoFlow, to learn optical flow for panoramic images. To overcome the distortions introduced by equirectangular projection in panoramic transformation, we design a Flow Distortion Augmentation (FDA) method, which contains radial flow distortion (FDA-R) or equirectangular flow distortion (FDA-E). We further look into the definition and properties of cyclic optical flow for panoramic videos, and hereby propose a Cyclic Flow Estimation (CFE) method by leveraging the cyclicity of spherical images to infer 360° optical flow and converting large displacement to relatively small displacement. PanoFlow is applicable to any existing flow estimation method and benefits from the progress of narrow-FoV flow estimation. In addition, we create and release a synthetic panoramic dataset FlowScape based on CARLA to facilitate training and quantitative analysis. PanoFlow achieves state-of-the-art performance on the public OmniFlowNet and the established FlowScape benchmarks. Our proposed approach reduces the End-Point-Error (EPE) on FlowScape by 27.3%. On OmniFlowNet, PanoFlow achieves a 55.5% error reduction from the best published result. We also qualitatively validate our method via a collection vehicle and a public real-world OmniPhotos dataset, indicating strong potential and robustness for real-world navigation applications. Code and dataset are publicly available at https://github.com/MasterHow/PanoFlow.
Semi-SMD: Semi-Supervised Metric Depth Estimation via Surrounding Cameras for Autonomous Driving
Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)(机器人与自主系统方向,香港科学与技术大学(广州));Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology(电子与计算机工程系,香港科学与技术大学)
Intention-aware Long Horizon Trajectory Prediction of Surrounding Vehicles using Dual LSTM Networks
California PATH, University of California, Berkeley(加州大学伯克利分校加州合作伙伴先进运输技术项目(PATH));Tsinghua University(清华大学);University of California, Berkeley(加州大学伯克利分校)
Dynamic Risk Assessment for Autonomous Vehicles from Spatio-Temporal Probabilistic Occupancy Heatmaps
Department of Civil and Environmental Engineering, University of California, Berkeley, CA, USA(加州大学伯克利分校土木与环境工程系);D3 Analytics Innovation, Allstate Insurance Company, Northbrook, IL, USA(Allstate保险公司D3 Analytics创新部门);Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA(加州大学伯克利分校电气工程与计算机科学系)
Multi-scale Temporal Fusion Transformer for Incomplete Vehicle Trajectory Prediction
University of California, Los Angeles (UCLA)(加利福尼亚大学洛杉矶分校);FHWA Center of Excellence on New Mobility and Automated Vehicles(联邦公路管理局新 mobility 与自动驾驶汽车卓越中心)
Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment
Indian Institute of Science(印度科学学院);Indian Institute of Science Education and Research, Bhopal(博帕尔印度科学教育与研究学院);Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science(印度科学学院罗伯特·博世网络物理系统中心)
SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation
Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua University(清华大学);Tianjin University(天津大学);PhiGent Robotics(鉴智机器人)
Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
School of Computer Science, China University of Geosciences(中国地质大学(武汉)计算机学院);Engineering Research Center of Natural Resource Information Management and Digital Twin Engineering Software, Ministry of Education(教育部自然资源信息管理与数字孪生工程软件工程研究中心);Second Surveying and Mapping Institute of Hunan Province(湖南省第二测绘院);Key Laboratory of Natural Resources Monitoring and Supervision in Southern Hilly Region, Ministry of Natural Resources(自然资源部南方山地自然资源监测与监管重点实验室)