Reconstruct from BEV: A 3D Lane Detection Approach based on Geometry Structure Prior
SenseTime Research（商汤科技研究院）；Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；University of Tuebingen（图宾根大学）；Shanghai AI Laboratory（上海人工智能实验室）
Chenguang Li, Jia Shi, Ya Wang, Guangliang Cheng
备注：
Proceedings of the CVPR 2022 Workshop of Autonomous Driving
In this paper, we propose an advanced approach in targeting the problem of monocular 3D lane detection by leveraging geometry structure underneath the process of 2D to 3D lane reconstruction. Inspired by previous methods, we first analyze the geometry heuristic between the 3D lane and its 2D representation on the ground and propose to impose explicit supervision based on the structure prior, which makes it achievable to build inter-lane and intra-lane relationships to facilitate the reconstruction of 3D lanes from local to global. Second, to reduce the structure loss in 2D lane representation, we directly extract BEV lane information from front view images, which tremendously eases the confusion of distant lane features in previous methods. Furthermore, we propose a novel task-specific data augmentation method by synthesizing new training data for both segmentation and reconstruction tasks in our pipeline, to counter the imbalanced data distribution of camera pose and ground slope to improve generalization on unseen data. Our work marks the first attempt to employ the geometry prior information into DNN-based 3D lane detection and makes it achievable for detecting lanes in an extra-long distance, doubling the original detection range. The proposed method can be smoothly adopted by other frameworks without extra costs. Experimental results show that our work outperforms state-of-the-art approaches by 3.8% F-Score on Apollo 3D synthetic dataset at real-time speed of 82 FPS without introducing extra parameters.
3D lane detection which plays a crucial role in vehicle routing, has recently been a rapidly developing topic in autonomous driving. Previous works struggle with practicality due to their complicated spatial transformations and inflexible representations of 3D lanes. Faced with the issues, our work proposes an efficient and robust monocular 3D lane detection called BEV-LaneDet with three main contributions. First, we introduce the Virtual Camera that unifies the in/extrinsic parameters of cameras mounted on different vehicles to guarantee the consistency of the spatial relationship among camer…
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；TuSimple（图森未来）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Monocular 3D lane detection is a challenging task due to its lack of depth information. A popular solution is to first transform the front-viewed (FV) images or features into the bird-eye-view (BEV) space with inverse perspective mapping (IPM) and detect lanes from BEV features. However, the reliance of IPM on flat ground assumption and loss of context information make it inaccurate to restore 3D information from BEV representations. An attempt has been made to get rid of BEV and predict 3D lanes from FV representations directly, while it still underperforms other BEV-based methods given its l…
Anchor3DLane++: 3D Lane Detection via Sample-Adaptive Sparse 3D Anchor Regression
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；TuSimple（图森未来）；The Chinese University of Hong Kong（香港中文大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Sun Yat-sen University（中山大学）；Shenzhen Loop Area Institute（深圳河套学院）；Baidu Inc.（百度公司）；Harbin Institute of Technology（哈尔滨工业大学）；Pengcheng Laboratory（鹏城实验室）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）
NullMax；School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）
3D lane detection is an integral part of autonomous driving systems. Previous CNN and Transformer-based methods usually first generate a bird's-eye-view (BEV) feature map from the front view image, and then use a sub-network with BEV feature map as input to predict 3D lanes. Such approaches require an explicit view transformation between BEV and front view, which itself is still a challenging problem. In this paper, we propose CurveFormer, a single-stage Transformer-based method that directly calculates 3D lane parameters and can circumvent the difficult view transformation step. Specifically,…
SparseLaneSTP: Leveraging Spatio-Temporal Priors with Sparse Transformers for 3D Lane Detection
Bosch Mobility Solutions, Robert Bosch GmbH（博世移动解决方案，罗伯特·博世有限公司）；Institute of Neuro- and Bioinformatics, University of Lübeck（神经与生物医学研究所，吕贝克大学）；Institute for Signal Processing and System Theory, University of Stuttgart（信号处理与系统理论研究所，斯图加特大学）
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime Research（商汤科技研究院）
Methods for 3D lane detection have been recently proposed to address the issue of inaccurate lane layouts in many autonomous driving scenarios (uphill/downhill, bump, etc.). Previous work struggled in complex cases due to their simple designs of the spatial transformation between front view and bird's eye view (BEV) and the lack of a realistic dataset. Towards these issues, we present PersFormer: an end-to-end monocular 3D lane detector with a novel Transformer-based spatial feature transformation module. Our model generates BEV features by attending to related front-view local regions with ca…
In this paper, we propose a method for online extrinsic camera calibration, i.e., estimating pitch, yaw, roll angles and camera height from road surface in sequential driving scene images. The proposed method estimates the extrinsic camera parameters in two steps: 1) pitch and yaw angles are estimated simultaneously using a vanishing point computed from a set of lane boundary observations, and then 2) roll angle and camera height are computed by minimizing difference between lane width observations and a lane width prior. The extrinsic camera parameters are sequentially updated using extended …
General Motors Advanced Technical Center Israel（通用汽车以色列先进技术中心）
We propose a novel camera-based DNN method for 3D lane detection with uncertainty estimation. Our method is based on a semi-local, BEV, tile representation that breaks down lanes into simple lane segments. It combines learning a parametric model for the segments along with a deep feature embedding that is then used to cluster segment together into full lanes. This combination allows our method to generalize to complex lane topologies, curvatures and surface geometries. Additionally, our method is the first to output a learning based uncertainty estimation for the lane detection task. The effic…
We present ONCE-3DLanes, a real-world autonomous driving dataset with lane layout annotation in 3D space. Conventional 2D lane detection from a monocular image yields poor performance of following planning and control tasks in autonomous driving due to the case of uneven road. Predicting the 3D lane layout is thus necessary and enables effective and safe driving. However, existing 3D lane detection datasets are either unpublished or synthesized from a simulated environment, severely hampering the development of this field. In this paper, we take steps towards addressing these issues. By exploi…
Cyprus International University（塞浦路斯国际大学）；the Hong Kong University of Science and Technology（香港科技大学）；Hangzhou ATG Intelligent Technology Co. Ltd.（杭州安智智能科技有限公司）；Guolu Gaoke Engineering Technology Institute Co. Ltd.（国路高科工程技术研究院有限公司）
In this paper, a robust lane detection algorithm is proposed, where the vertical road profile of the road is estimated using dynamic programming from the v-disparity map and, based on the estimated profile, the road area is segmented. Since the lane markings are on the road area and any feature point above the ground will be a noise source for the lane detection, a mask is created for the road area to remove some of the noise for lane detection. The estimated mask is multiplied by the lane feature map in a bird's eye view (BEV). The lane feature points are extracted by using an extended versio…
Institute for Artificial Intelligence at Tsinghua University（清华大学人工智能学院）；State Key Laboratory of Intelligent Technology and Systems（智能技术与系统国家重点实验室）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Center for Intelligent Connected Vehicles and Transportation（智能网联汽车与交通中心）；Department of Computer Sicence and Technology, Tsinghua University（清华大学计算机科学与技术系）
In a complex road traffic scene, illegal lane intrusion of pedestrians or cyclists constitutes one of the main safety challenges in autonomous driving application. In this paper, we propose a novel object-level phase space reconstruction network (PSRNet) for motion time series classification, aiming to recognize lane intrusion actions that occur 150m ahead through a monocular camera fixed on moving vehicle. In the PSRNet, the movement of pedestrians and cyclists, specifically viewed as an observable object-level dynamic process, can be reconstructed as trajectories of state vectors in a latent…
Semantics-Guided Multimodal Masked Autoencoder Pretraining for 3D BEV Object Detection
University of Lincoln, Lincoln Centre for Autonomous Systems（林肯大学，林肯自主系统中心）；University of Cambridge, Institute for Manufacturing, Department of Engineering（剑桥大学，制造研究所，工程系）
HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to expensive sensors and time-consuming computation. Camera-based methods usually need to perform road segmentation and view transformation separately, which often causes distortion and missing content. To push the limits of the technology, we present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view given a front-view monocular image only. We propose a front-to-top view projection (FTVP) module, which takes the constraint of cycle consis…
Autonomous detection of lane markers improves road safety, and purely visual tracking is desirable for widespread vehicle compatibility and reducing sensor intrusion, cost, and energy consumption. However, visual approaches are often ineffective because of a number of factors; e.g., occlusion, poor weather conditions, and paint wear-off. We present an approach to enhance lane marker appearance for assisted and autonomous driving, particularly under poor visibility. Our method, named SafeDrive, attempts to improve visual lane detection approaches in drastically degraded visual conditions. SafeD…
GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Robbyant；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；The Hong Kong University of Science and Technology（香港科技大学）
Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection
School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；SenseTime Research（商汤科技研究院）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；State Key Laboratory of Complex & Critical Software Environment, Beihang University（北京航空航天大学复杂与关键软件环境国家重点实验室）
Collaborative Perceiver: Elevating Vision-based 3D Object Detection via Local Density-Aware Spatial Occupancy
The Open Distributed Systems (ODS) Group at the Technische Universität Berlin（技术大学柏林开放分布式系统（ODS）小组）；Fraunhofer FOKUS (Berlin, Germany)（弗劳恩霍夫研究所FOKUS（柏林，德国））
Variational autoencoder (VAE) has widely been utilized for modeling data distributions because it is theoretically elegant, easy to train, and has nice manifold representations. However, when applied to image reconstruction and synthesis tasks, VAE shows the limitation that the generated sample tends to be blurry. We observe that a similar problem, in which the generated trajectory is located between adjacent lanes, often arises in VAE-based trajectory forecasting models. To mitigate this problem, we introduce a hierarchical latent structure into the VAE-based forecasting model. Based on the a…
OmniGen: Unified Multimodal Sensor Generation for Autonomous Driving
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Westlake University（西湖大学）；Li Auto Inc.（Li汽车公司）；The University of Toronto（多伦多大学）；University of Macau（澳门大学）
